Back
Stanford HAI: The Disinformation Machine
webCredibility Rating
4/5
High(4)High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: Stanford HAI
Published by Stanford HAI, this piece is relevant to AI safety discussions around misuse risks, particularly how large language models can be weaponized for influence operations and the societal harms that may result from unchecked AI content generation.
Metadata
Importance: 52/100news articleanalysis
Summary
Stanford HAI examines the growing threat of AI-generated disinformation and propaganda, exploring how susceptible individuals and societies are to algorithmically produced misleading content. The piece investigates the mechanisms by which AI systems can generate and amplify false narratives, and considers implications for democratic processes and public trust.
Key Points
- •AI systems can generate highly convincing disinformation at scale, making it harder to distinguish from authentic human-produced content.
- •Research suggests people may be more susceptible to AI-generated propaganda than previously assumed, with limited ability to detect synthetic text.
- •The scale and speed of AI content generation poses novel challenges for existing fact-checking and content moderation infrastructure.
- •Mitigation strategies may include watermarking AI content, improved media literacy, and policy-level interventions on AI deployment.
- •The intersection of AI capabilities and information warfare raises significant concerns for electoral integrity and societal cohesion.
Cached Content Preview
HTTP 200Fetched Apr 9, 20268 KB
The Disinformation Machine: How Susceptible Are We to AI Propaganda? | Stanford HAI Skip to content
Navigate
About
Events
AI Glossary
Careers
Search
Participate
Get Involved
Support HAI
Contact Us
Stay Up To Date
Get the latest news, advances in research, policy work, and education program updates from HAI in your inbox weekly.
Sign Up For Latest News
iStock With a bit of prodding, AI-generated propaganda is more effective than propaganda written by humans.
Michael Tomz , a professor of political science at Stanford School of Humanities and Sciences and faculty affiliate at the Stanford Institute for Human-Centered AI (HAI), recently gave a talk in Taiwan about the use of AI to generate propaganda. That morning, he recalled, he saw a headline in the Taipei Times reporting that the Chinese government was using AI-generated social media posts to influence voters in Taiwan and the United States.
“That very day, the newspaper was documenting the Chinese government doing exactly what I was presenting on,” Tomz said.
AI propaganda is here. But is it persuasive? Recent research published in PNAS Nexus and conducted by Tomz, Josh Goldstein from the Center for Security and Emerging Technology at Georgetown University, and three Stanford colleagues—master’s student Jason Chao , research scholar Shelby Grossman , and lecturer Alex Stamos —examined the effectiveness of AI-generated propaganda.
They found, in short, that it works.
When Large Language Models Lie
The researchers conducted an experiment, funded by HAI, in which participants were assigned to one of three groups.
Read the full study, How persuasive is AI-generated propaganda?
The first group, the control, read a series of thesis statements on subjects that known propagandists want people to believe. “Most U.S. drone strikes in the Middle East have targeted civilians, rather than terrorists,” for instance. Or, “Western sanctions have led to a shortage of medical supplies in Syria.” Because this group read only these statements and no propaganda related to them, it provided the researchers with a baseline understanding of how many people believe these claims. The second group of participants read human-crafted propaganda that was written on the subject of the thesis statements and later uncovered by investigative journalists or researchers. The third group was given propaganda on the same topics as groups one and two that had been generated by large language model GPT-3.
The researchers found that about one quarter of the control group agreed with the thesis statements without reading any propaganda about them. Propaganda written by humans bumped this up to 47 percent, and propaganda written by AI to 43 percent.
“We then tweaked the AI process by adding humans into the loop,” Tomz said. “We had people edit the input—the text used to prompt the language model—and curate the output by discarding articles that failed to ma
... (truncated, 8 KB total)Resource ID:
cfd7b21d0ae4298d | Stable ID: sid_y9xEKVmNJX