In a recent development that has sent ripples through the scientific community, researchers have identified instances of dishonest utilization of ChatGPT, a popular AI chatbot, in scientific papers. These revelations highlight a growing concern about the transparency and integrity of research publications.
The discovery
On August 9th, the journal *Physica Scripta* published a paper that aimed to unravel complex mathematical equations. However, eagle-eyed computer scientist Guillaume Cabanac noticed an odd phrase on the third page of the manuscript: ‘Regenerate response.’ This phrase turned out to be a label from ChatGPT, an AI chatbot used for generating text in response to user prompts. Cabanac promptly shared his findings on PubPeer, a platform where scientists discuss published research.
Subsequent investigations revealed that the authors of the paper had indeed employed ChatGPT to assist in drafting their manuscript. Remarkably, this anomaly had gone unnoticed during the two-month peer review process and the typesetting phase. As a result, the publisher, IOP Publishing, decided to retract the paper due to the authors’ failure to declare their use of the tool during submission.
The wider issue
This case is not an isolated incident. Since April, Guillaume Cabanac has identified more than a dozen journal articles containing the telltale ChatGPT phrases ‘Regenerate response’ or ‘As an AI language model, I…,’ all posted on PubPeer. Many publishers, including Elsevier and Springer Nature, permit the use of ChatGPT and similar language models as long as authors declare it.
However, these instances of detection represent only the tip of the iceberg. Researchers suspect that numerous undisclosed peer-reviewed papers have utilized ChatGPT without proper declaration. The phrases themselves have evolved, with ChatGPT changing its ‘Regenerate response’ button to ‘Regenerate’ in an update.
Undisclosed usage across journals
Guillaume Cabanac has uncovered typical ChatGPT phrases in papers published in Elsevier journals. In one case, a paper published in *Resources Policy* on August 3rd explored the impact of e-commerce on fossil-fuel efficiency in developing countries. Cabanac noticed inconsistencies in the equations, but the giveaway was a statement above a table: ‘Please note that as an AI language model, I am unable to generate specific tables or conduct tests…’
Elsevier has acknowledged the issue and is currently investigating it. The authors of the paper, affiliated with Liaoning University in Shenyang, China, and the Chinese Academy of International Trade and Economic Cooperation in Beijing, have not responded to requests for comment.
The challenge of detecting deception
Papers partly or wholly written by computer software without disclosure are not new, but they typically exhibit distinguishable traces such as language patterns or mistranslated phrases. Deleting the boilerplate ChatGPT phrases, however, renders the AI-generated text nearly impossible to spot. This creates an ongoing challenge for the scientific community, often described as an “arms race” between scammers and those trying to maintain research integrity.
Impact on peer review
Undisclosed use of ChatGPT has also surfaced in peer-reviewed conference papers and preprints. Authors, when confronted on PubPeer, have sometimes admitted to using ChatGPT without declaration. This revelation highlights a significant issue – overburdened peer reviewers may not have sufficient time to thoroughly scrutinize manuscripts for red flags.
The rise of fake manuscripts
The proliferation of generative AI tools like ChatGPT raises concerns about the potential rise of paper mills, companies that produce and sell fake manuscripts to researchers seeking to bolster their publishing output. This phenomenon threatens to exacerbate an already challenging problem in academic publishing.
Spotting the deception
Language models like ChatGPT have been known to generate false references, a potential signal for peer reviewers attempting to detect their use in manuscripts. Fictitious references have been identified in some cases, serving as red flags to alert reviewers. Vigilance in verifying references is becoming increasingly important.
The discovery of undisclosed ChatGPT usage in scientific publications underscores the importance of transparency and integrity in research. As the scientific community grapples with the challenges posed by evolving AI tools, it is evident that stricter guidelines and heightened vigilance in peer review are essential to maintain the credibility of academic publishing. Addressing these issues is paramount to ensuring that the pursuit of knowledge remains grounded in ethical principles and rigorous research practices.
