In a recent study conducted by researchers from Stanford University and Georgetown University, revelations have emerged regarding Facebook’s recommendation algorithms. The study highlights how these algorithms are inadvertently boosting AI-generated images posted by spammers and scammers, leading to a proliferation of deceptive content on the platform.
AI-generated images: A rising threat
The study highlights the alarming trend of AI-generated images, often perplexing or captivating to users, being utilized as part of larger spam or scam campaigns on Facebook. One example cited in the research is the phenomenon of ‘Shrimp Jesus,’ an AI-created image that has gained viral traction multiple times on the platform. Originally intended to amass followers, these images have evolved to drive traffic toward websites filled with advertisements or peddling subpar products.
The concerning aspect of these AI-generated images lies in their ability to reach unsuspecting users who do not follow the originating pages. Many users engage with these images without realizing they interact with fabricated content. While some images, like ‘Shrimp Jesus,’ may be blatantly fantastical, others portraying everyday objects or artwork can deceive users about their artificial nature, leading to widespread misinformation and exploitation.
Regulatory response and scrutiny
Regulatory authorities are taking action as the threat posed by AI-generated spam and scams becomes increasingly apparent. The UK’s Advertising Standards Authority (ASA) is currently scrutinizing the application of generative AI in advertising, with potential implications for new guidelines to be introduced this year. Similarly, the US Federal Trade Commission has intensified its focus on the issue, recognizing the urgent need to address deceptive practices on social media platforms.
In 2021, Norway implemented revised laws mandating advertisers and influencers to disclose the use of digitally manipulated images. This regulation, aimed at enhancing transparency and consumer trust, could also extend to AI-generated advertisements.
The proliferation of AI-generated spam and scams on Facebook underscores the need for robust regulatory measures and platform accountability to safeguard users from deceptive content.
As researchers continue to uncover the extent of this phenomenon, social media companies must prioritize the integrity of their recommendation algorithms and combat the exploitation of generative AI for malicious purposes. We can only mitigate the risks posed by AI-generated deception through concerted efforts from regulatory bodies and tech platforms and ensure a safer online environment for all users.
