Is this AI or a journalist? Research reveals stylistic differences in news articles

News articles produced by Artificial Intelligence (AI) do not have the same creative flair as stories written by human journalists, according to research into the stylistic differences between the storytellers. 

Charles Darwin University (CDU) researchers sought to find the linguistic difference between journalist-written and AI-generated news articles. 

Researchers fed 150 news articles, on topics such as politics, sports, military affairs, and technology, into chatbot Gemini and prompted it to compose 150 articles aligned with the content of the human-written stories. 

The human-written stories were sourced from the New York Times, the Sydney Morning Herald, and the Australian. 

CDU Master of Information Technology graduate Van Hieu Tran, who is the lead investigator of this study, said while AI-generated content was becoming more sophisticated each day, there were still subtle stylistic differences. 

The researchers found human journalists have greater variety in sentence and paragraph length, while Gemini had a lack of variability in syntax. 

Human journalists also use more verbs, suggesting their writing style focuses more on explaining actions to engage readers. Gemini in comparison uses more nouns than verbs. 

“Our paper suggests that AI and human writers produce equally readable content,” Mr Tran said. 

“However, the paper also finds that human writers produce more diverse syntactic and paragraph structures in their journalistic pieces than AI did. AI produces ‘more boring’ content that lacks stylistic diversity and writers' unique flair.

CDU Lecturer in Information Technology Dr Yakub Sebastian stresses that this research is important because it suggests that human ingenuity and deep personality could still thrive, and potentially more appealing to human readers.

“There is also a deeper question as to whether it matters whether we could distinguish AI versus human writers beyond the issue of attribution/originality especially if all facts in the news are equally accurate. 

“We think it matters because news often shapes opinions and narratives, not just delivering facts. AI biases, for instance, are certainly one thing that we need to be concerned about.” 

Dr Sebastian, who supervised the research and co-authored its publication, said this type of research was important given how quickly AI is evolving to resemble human copy, and said the knowledge gained from this paper could have practical applications. 

“AI models advance at a breakneck speed, and we see them increasingly capable of doing what humans can do,” Dr Sebastian said.

“As such, we can expect that distinguishing between human-generated and AI-generated text will become increasingly difficult for human readers. This is already happening. Just recently, an Italian newspaper officially published the world's first AI-generated newspapers.”

“We can think of a Turnitin-equivalent Web browser plug-in that could flag with a certain probability that the news being displayed on the Web browser is AI-generated based on the machine learning model we developed in this paper.”

Distinguishing Human Journalists from Artificial Storytellers Through Stylistic Fingerprints was published in Emerging Trends in Machine Learning and Artificial Intelligence, a special issue of the international journal Computers.