Mesa-Optimization
AccidentCatastrophicMesa-optimization occurs when a learned model (like a neural network) is itself an optimizer. The "mesa-" prefix means the optimization emerges from within the training process, as opposed to the "base" optimizer (the training algorithm itself).
Severity
Catastrophic
Likelihood
Medium (theoretical)
Time Horizon
~2035
Maturity
Growing
Full Wiki Article
Read the full wiki article for detailed analysis, background, and references.
Read wiki article →Related Entities3
organization
Sources3
Assessment
SeverityCatastrophic
LikelihoodMedium (theoretical)
Time Horizon~2035
MaturityGrowing
CategoryAccident
Details
Coined ByHubinger et al.
Key PaperRisks from Learned Optimization (2019)
Tags
inner-alignmentouter-alignmentdeceptionlearned-optimizationbase-optimizer