PNAS study from December 2024
webAuthors
Credibility Rating
Gold standard. Rigorous peer review, high editorial standards, and strong institutional reputation.
Rating inherited from publication venue: PNAS
Empirical RCT evidence directly relevant to AI deployment risks in high-stakes information environments; important for policymakers and researchers evaluating real-world impacts of AI systems on public epistemics and misinformation.
Paper Details
Metadata
Summary
A preregistered randomized controlled experiment published in PNAS (2024) finds that despite 90% accuracy in identifying false headlines, LLM-generated fact checks do not improve users' ability to discern headline accuracy or promote accurate news sharing—unlike human fact checks. Critically, LLM fact checks caused harm in specific failure modes: reducing belief in true headlines mislabeled as false and increasing belief in false headlines when the AI expressed uncertainty.
Key Points
- •LLMs achieved 90% accuracy identifying false headlines, but this did not translate into improved user discernment or accurate news-sharing behavior.
- •Human-generated fact checks significantly improved discernment; LLM fact checks did not, revealing a human-AI trust gap.
- •AI uncertainty expressions backfired: users were more likely to believe false headlines when the LLM expressed doubt about their veracity.
- •When users chose to view LLM fact checks, they became more likely to share both true and false news, and more likely to believe false headlines.
- •Findings highlight systemic risks of deploying AI fact-checking at scale without policies to mitigate unintended harmful effects on information integrity.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| AI-Era Epistemic Infrastructure | Approach | 59.0 |
Cached Content Preview
# Fact-checking information from large language models can decrease headline discernment Authors: Matthew R. DeVerna, Harry Yaojun Yan, Kai-Cheng Yang, Filippo Menczer Journal: Proceedings of the National Academy of Sciences Published: 2024-12-10 DOI: 10.1073/pnas.2322823121 ## Abstract Fact checking can be an effective strategy against misinformation, but its implementation at scale is impeded by the overwhelming volume of information online. Recent AI language models have shown impressive ability in fact-checking tasks, but how humans interact with fact-checking information provided by these models is unclear. Here, we investigate the impact of fact-checking information generated by a popular large language model (LLM) on belief in, and sharing intent of, political news headlines in a preregistered randomized control experiment. Although the LLM accurately identifies most false headlines (90%), we find that this information does not significantly improve participants’ ability to discern headline accuracy or share accurate news. In contrast, viewing human-generated fact checks enhances discernment in both cases. Subsequent analysis reveals that the AI fact-checker is harmful in specific cases: It decreases beliefs in true headlines that it mislabels as false and increases beliefs in false headlines that it is unsure about. On the positive side, AI fact-checking information increases the sharing intent for correctly labeled true headlines. When participants are given the option to view LLM fact checks and choose to do so, they are significantly more likely to share both true and false news but only more likely to believe false headlines. Our findings highlight an important source of potential harm stemming from AI applications and underscore the critical need for policies to prevent or mitigate such unintended consequences.
246011d53fc7e0d5 | Stable ID: sid_A5hCTAe94G