Back
What mistakes has the AI safety movement made?
webAuthor
EuanMcLean
Credibility Rating
3/5
Good(3)Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.
Rating inherited from publication venue: EA Forum
A useful critical perspective on the AI safety field's strategic and cultural shortcomings, based on semi-structured interviews; valuable for understanding internal debates about movement effectiveness and direction.
Forum Post Details
Karma
66
Comments
3
Forum
eaforum
Forum Tags
AI safetyCause prioritizationExistential riskCollections and resourcesCriticism of effective altruismOpinion
Part of sequence: Big Picture AI Safety
Metadata
Importance: 62/100blog postanalysis
Summary
A qualitative synthesis of interviews with 17 AI safety experts identifying systemic mistakes in the AI safety movement, including overreliance on abstract reasoning, insularity, counterproductive messaging, and neglect of policy pathways. The post provides rare critical self-reflection from within the community about strategic and epistemic failures. Some interviewees questioned whether the movement's overall track record has been net positive.
Key Points
- •Overreliance on abstract theoretical arguments over empirical approaches has limited the movement's credibility and effectiveness.
- •Insularity and groupthink within AI safety circles have reduced independent thinking and suppressed dissenting perspectives.
- •Off-putting or alarmist messaging has alienated mainstream researchers, policymakers, and potential allies.
- •Close relationships with leading AGI labs may compromise the movement's independence and ability to advocate for safety.
- •Insufficient attention to policy and public outreach has left important safety levers underutilized.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Frontier Model Forum | Organization | 58.0 |
Cached Content Preview
HTTP 200Fetched Apr 10, 202624 KB
# What mistakes has the AI safety movement made?
By EuanMcLean
Published: 2024-05-23
This is the third of three posts summarizing what I learned when I interviewed 17 AI safety experts about their "big picture" of the existential AI risk landscape: how AGI will play out, how things might go wrong, and what the AI safety community should be doing. See [here](https://forum.effectivealtruism.org/posts/uJioXCz5Foo9eqpJ9/big-picture-ai-safety-introduction) for a list of the participants and the standardized list of questions I asked.
This post summarizes the responses I received from asking “Are there any big mistakes the AI safety community has made in the past or are currently making?”
A rough decompositions of the main themes brought up. The figures omit some less popular themes, and double-count respondents who brought up more than one theme.
> “*Yeah, probably most things people are doing are mistakes. This is just some random group of people. Why would they be making good decisions on priors? When I look at most things people are doing, I think they seem not necessarily massively mistaken, but they seem somewhat confused or seem worse to me by like 3 times than if they understood the situation better.*” \- Ryan Greenblatt
> *“If we look at the track record of the AI safety community, it quite possibly has been harmful for the world.*” \- Adam Gleave
> “[*Longtermism*](https://www.effectivealtruism.org/articles/longtermism)* was developed basically so that AI safety could be the most important cause by the utilitarian EA calculus. That's my take.*” \- Holly Elmore
Participants pointed to a range of mistakes they thought the AI safety movement had made. Key themes included an overreliance on theoretical argumentation, being too insular, putting people off by pushing weird or extreme views, supporting the leading AGI companies, insufficient independent thought, advocating for an unhelpful pause to AI development, and ignoring policy as a potential route to safety.
How to read this post
---------------------
This is not a scientific analysis of a systematic survey of a representative sample of individuals, but my qualitative interpretation of responses from a loose collection of semi-structured interviews. Take everything here with the appropriate seasoning.
Results are often reported in the form “*N* respondents held view *X*”. This does **not** imply that “17-*N* respondents disagree with view *X*”, since not all topics, themes and potential views were addressed in every interview. What “*N* respondents held view *X*” tells us is that at least *N* respondents hold *X*, and consider the theme of *X* important enough to bring up.
The following is a summary of the main themes that came up in my interviews. Many of the themes overlap with one anothe
... (truncated, 24 KB total)Resource ID:
8aec760a9e7e4c93 | Stable ID: sid_GreDAGZFBQ