MIT persuasion study
paperAuthors
Credibility Rating
Gold standard. Rigorous peer review, high editorial standards, and strong institutional reputation.
Rating inherited from publication venue: Science
Empirical study examining humans' ability to distinguish AI-generated content (GPT-3) from authentic information and detect disinformation, directly relevant to understanding AI capabilities in content generation and information integrity risks.
Paper Details
Metadata
Abstract
Artificial intelligence (AI) is changing the way we create and evaluate information, and this is happening during an infodemic, which has been having marked effects on global health. Here, we evaluate whether recruited individuals can distinguish disinformation from accurate information, structured in the form of tweets, and determine whether a tweet is organic or synthetic, i.e., whether it has been written by a Twitter user or by the AI model GPT-3. The results of our preregistered study, including 697 participants, show that GPT-3 is a double-edge sword: In comparison with humans, it can produce accurate information that is easier to understand, but it can also produce more compelling disinformation. We also show that humans cannot distinguish between tweets generated by GPT-3 and written by real Twitter users. Starting from our results, we reflect on the dangers of AI for disinformation and on how information campaigns can be improved to benefit global health.
Summary
This MIT study examined whether humans can distinguish between accurate and false information in tweets, and whether they can identify AI-generated content from GPT-3 versus human-written tweets. With 697 participants, researchers found that GPT-3 presents a dual challenge: it can produce accurate, easily understandable information but also generates more compelling disinformation. Critically, humans cannot reliably distinguish between GPT-3-generated and human-written tweets, raising significant concerns about AI's potential to spread disinformation during an infodemic.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| AI Capability Threshold Model | Analysis | 72.0 |
Cached Content Preview
# AI model GPT-3 (dis)informs us better than humans Authors: Giovanni Spitale, Nikola Biller-Andorno, Federico Germani Journal: Science Advances Published: 2023-06-30 DOI: 10.1126/sciadv.adh1850 ## Abstract Artificial intelligence (AI) is changing the way we create and evaluate information, and this is happening during an infodemic, which has been having marked effects on global health. Here, we evaluate whether recruited individuals can distinguish disinformation from accurate information, structured in the form of tweets, and determine whether a tweet is organic or synthetic, i.e., whether it has been written by a Twitter user or by the AI model GPT-3. The results of our preregistered study, including 697 participants, show that GPT-3 is a double-edge sword: In comparison with humans, it can produce accurate information that is easier to understand, but it can also produce more compelling disinformation. We also show that humans cannot distinguish between tweets generated by GPT-3 and written by real Twitter users. Starting from our results, we reflect on the dangers of AI for disinformation and on how information campaigns can be improved to benefit global health.
9e3c9400f4428304 | Stable ID: sid_UTAXS3SmsM