Artificial intelligence (AI) chatbots, fueled by generative AI models, have been inadvertently led astray by a research blog post. The incident highlights the need for enhanced safeguards as AI-enhanced search engines become more prevalent.
Hallucinations in AI chatbots
Information science researcher Daniel S. Griffin inadvertently triggered a unique conundrum in the realm of AI chatbots when he posted examples of false information on his blog. His aim was to illustrate instances of AI chatbots disseminating misinformation. Griffin showcased two examples related to the influential computer scientist Claude E. Shannon, with a clear disclaimer indicating that the chatbots’ information was untrue.
However, this disclaimer proved insufficient in preventing machine scrapers from indexing the false information. Griffin was startled to discover that multiple chatbots, including Microsoft’s Bing and Google’s Bard, had referenced the hallucinated data as if it were factual, ranking it at the top of their search results. When users queried the chatbots about specific details regarding Shannon, the bots relied on Griffin’s disclaimer to construct a consistent yet erroneous narrative. This resulted in attributing a paper to Shannon that he had never authored.
Furthermore, the concerning aspect of this situation was the absence of any indication in Bing and Bard’s search results that their sources were derived from Language Model AI (LLM). This phenomenon closely mirrors cases where individuals paraphrase or quote sources out of context, leading to flawed research. However, the Griffin incident demonstrated that generative AI models could potentially automate this mistake on a much larger scale.
The consequences of AI misinformation
The ramifications of this incident extend beyond mere research inaccuracies. Microsoft has since corrected the error in Bing and suggested that the issue is more likely to arise in subjects with limited human-written content online. This vulnerability opens the door to a disturbing possibility – a theoretical blueprint for malicious actors to intentionally weaponize LLMs to propagate misinformation by manipulating search engine results. This tactic bears resemblance to past instances where hackers have used fraudulent websites to distribute malware by securing top search result rankings.
This incident echoes a prior warning from June, which posited that as the internet becomes increasingly saturated with LLM-generated content, it will be utilized to train future LLMs. This feedback loop could lead to a significant deterioration in the quality and trustworthiness of AI models, a phenomenon referred to as “Model Collapse.” The need for vigilance in the development and utilization of AI models has never been more apparent.
The incident involving Daniel S. Griffin underscores the imperative for companies working with AI to prioritize the continual training of models with human-made content. Preserving less well-known information and material authored by minority groups can play a pivotal role in mitigating the proliferation of misinformation perpetuated by AI systems.
