A study found that when AI language models write tweets, people regard them to be more credible. At least, that’s how it looked in a recent research contrasting language produced by OpenAI’s GPT-3 model with content written by people.
Scientists polled people in the newest investigation to see if they could distinguish if a tweet were posted by a human or GPT-3. How so? Nobody could do it. Participants were also asked to rate the reliability of certain tweets.
Here’s where it gets complicated: the content at hand centered on scientific subjects like vaccines and climate change, which are frequently the target of online misinformation campaigns.
Participants in the study had a more difficult time identifying false material when it had been produced by the language model than when a human author had authored it. Better yet, if the material had been written by GPT-3 instead of a human, they were more likely to recognize it as being accurate.
The battle of AI vs. human minds
AI has become an internet contagion with Elon Musk asking developers to slow down on AI. However,no one has paid attention. Per the study, regardless of the veracity of the AI-generated data, participants in the study had more faith in GPT-3 than in other humans. This demonstrates the potential ability of AI language models to either inform or mislead the audience.
Study co-author and postdoctoral researcher at the University of Zurich’s Institute of Biomedical Ethics and History of Medicine Giovanni Spitale warns that “these amazing technologies could easily be used as weapons to create storms of false information on any topic you want.”
But, as Spitale points out, this does not have to be the case. There are ways to improve technology such that it is less likely to be used to spread disinformation. “It is neither intrinsically evil nor necessarily good. “It’s just an amplifier of human intention,” he explains.
Spitale and his colleagues compiled tweets about 11 different scientific issues, ranging from vaccines and covid-19 to climate change and evolution. They then instructed GPT-3 to create fresh tweets with either correct or incorrect information. In 2022, the team collected responses from 697 people online via Facebook advertising. They were largely from the United Kingdom, Australia, Canada, the United States, and Ireland, and they all spoke English. Their findings were published in the journal Science Advances today.
Unveiling disturbing trust in AI-generated deception
The study concluded that the information written by GPT-3 was “indistinguishable” from organic content. People polled couldn’t detect the difference. Note better, one of the study’s weaknesses is that the researchers cannot be assured that the tweets acquired from social media were not generated with the assistance of apps like ChatGPT.
There are some other limitations to this study to consider, such as the fact that participants had to analyze tweets out of context. They couldn’t look up the Twitter profile of whoever wrote the message, which would have helped them determine if it was a bot or not. Even viewing an account’s previous tweets and profile images could help one determine whether content associated with that account is false.
Participants excelled at spotting fake news when actual Twitter users posted it. Survey respondents were tricked slightly more successfully by tweets created by GPT-3 that contained false information. GPT-3 may not be the most compelling of the current crop of big language models. ChatGPT runs on the GPT-3.5 model, and users who wish access to the more recent GPT-4 model can pay a subscription fee.
This new study also discovered that survey respondents were better judges of correctness than GPT-3 in some circumstances. Similarly, the researchers instructed the language model to examine tweets and determine whether they were correct. When it comes to identifying accurate tweets, GPT-3 performed lower than human replies. Humans and GPT-3 performed similarly when it came to detecting misinformation.
For instance, if better training datasets were utilized to build language models, it might be more challenging for bad actors to exploit these technologies to mass-produce disinformation campaigns. When asked to produce false information by the researchers, GPT-3 “disobeyed” them on multiple occasions, most notably with regard to vaccines and autism. It’s possible that this is because training datasets had more information disproving conspiracy theories about those issues than they did about others.
According to Spitale, the best long-term method for combating disinformation is to promote critical thinking abilities so that people are better equipped to distinguish between facts and lies. And, given that regular people in the study appear to be as good or better assessors of accuracy than GPT-3, a little training could make them even better. According to the study, fact-checkers might work alongside language models like GPT-3 to boost legitimate public information efforts.
Don’t get me wrong: I love this technology […] I believe that narrative AIs will change the world… and it is up to us to decide whether or not it will be for the better.
Spitale
