Data-driven reporting is key for democratic participation. However, the field is becoming increasingly automated. Some journalists believe that such automated news production may reduce the attractiveness and comprehensibility of news articles for readers, and thus edit AI-generated texts before publication.
In the current study, researchers compared reader perception of AI-generated news stories that were post-edited by human journalists and those written entirely by journalists. They recruited 3, 135 UK online news consumers and asked them to evaluate 24 articles written with or without AI tools.
After conducting a factor analysis of the results, the researchers found that AI-generated articles were significantly less comprehensible than their manually-written counterparts. Further analysis showed that this might be partly explained by readers’ perceptions of other aspects; they rated numeric features, i.e. how the articles used data, alongside word choice, to be significantly worse in AI-generated articles than those that were entirely manually written.
The researchers noted, however, that despite the above variations, perceptions of news stories produced with AI did not significantly differ from those produced without in regards to sentence and paragraph length and writing style.
"Our results indicate the importance not only of maintaining human involvement in the automated production of data-driven news content, but of refining it," said lead author of the study, Sina Thäsler-Kordonouri, of the Institute for Communication Science and Media Research at Ludwig-Maximilians University, Munich, Germany, in a press release.
The researchers wrote that their findings also suggest how journalists could improve AI-generated texts.
“When ‘post-editing’ automated articles, journalists should aim to further reduce the quantity of numbers, better explain words that readers are unlikely to understand and change inappropriate wording,” they concluded in their paper.
Sources: Science Daily, Journalism