Surfshark’s ‘Bot or Not’ experiment involving over 700 participants shows that 47% struggle to correctly identify AI-generated comments from real human posts on simulated social platforms.
- 47% of participants failed to correctly identify more bots than humans.
- Emotional topics like immigration reduce bot detection accuracy significantly.
- Users under 40 outperform older age groups in distinguishing bots.
What happened
Cybersecurity firm Surfshark, together with Malmö University students, conducted an experiment titled ‘Bot or Not,’ putting 710 participants in a timed social media simulation to test their ability to distinguish AI bot comments from human ones. Participants viewed comments on four topics—two emotionally neutral and two politically charged—to measure detection accuracy under different discussion dynamics. Overall, only 53% correctly identified more bots than they missed, meaning nearly half failed to discern bots accurately.
The experiment revealed that participants performed best on non-emotional subjects like data centres, detecting 71% of bots with 76% accuracy, while detection rates dropped sharply on emotionally charged topics like immigration and women’s rights, with accuracy falling below 65%. Age also played a role, with participants under 40 demonstrating significantly better bot identification skills than older groups.
Why it matters
The study sheds light on the growing difficulty individuals face in detecting AI-driven disinformation online, especially in polarized or emotionally heated conversations. With bot accounts estimated to drive nearly a quarter of political discourse during election times and platforms removing billions of fake accounts annually, the challenge of distinguishing automated content poses serious risks for public debate integrity and misinformation spread.
Surfshark’s research emphasizes that the difficulty is not merely about critical reading ability but also about emotional influence on judgment. When topics trigger strong feelings, people’s mental filters for suspicious content weaken, allowing bots, which increasingly target such vulnerabilities, to blend more seamlessly into online conversations. This underscores the need for emotional awareness as part of digital literacy.
What to watch next
The ‘Bot or Not’ simulation remains available to the public online, offering users a practical tool to test their own skills and raise awareness about automated social media manipulation. As AI technology continues improving, future research and user education will be critical to building resilience against sophisticated bot-driven tactics that exploit emotional engagement.
Policymakers, platform operators, and cybersecurity experts should monitor how AI bots influence public opinion, especially around sensitive social issues. Adopting strategies that combine behavioral insights with technical detection methods may help mitigate the spread of deceptive automated content, while promoting digital literacy programs that emphasize emotional regulation and critical awareness in online environments.