Power-Seeking AI
AccidentCatastrophicPower-seeking refers to the tendency of optimal policies to acquire resources, influence, and capabilities beyond what's minimally necessary for their stated objective. Recent theoretical work has formalized when and why this occurs.
Severity
Catastrophic
Likelihood
Medium (theoretical)
Time Horizon
~2035
Maturity
Mature
Full Wiki Article
Read the full wiki article for detailed analysis, background, and references.
Read wiki article →Related Entities3
Sources3
Assessment
SeverityCatastrophic
LikelihoodMedium (theoretical)
Time Horizon~2035
MaturityMature
CategoryAccident
Details
Key PaperTurner et al. 2021
Tags
instrumental-convergenceself-preservationcorrigibilityoptimal-policiesresource-acquisition