LLM summaryLLM summaryBasic text summary used in search results, entity link tooltips, info boxes, and related page cards.crux content improve <id>ScheduleScheduleHow often the page should be refreshed. Drives the overdue tracking system.Set updateFrequency in frontmatterEntityEntityYAML entity definition with type, description, and related entries.Add entity YAML in data/entities/Edit historyEdit historyTracked changes from improve pipeline runs and manual edits.crux edit-log view <id>OverviewOverviewA ## Overview heading section that orients readers. Helps with search and AI summaries.
Tables1/ ~1TablesData tables for structured comparisons and reference material.Diagrams0DiagramsVisual content — Mermaid diagrams, charts, or Squiggle estimate models.Add Mermaid diagrams or Squiggle modelsInt. links19/ ~3Int. linksLinks to other wiki pages. More internal links = better graph connectivity.Ext. links0/ ~2Ext. linksLinks to external websites, papers, and resources outside the wiki.Add links to external sourcesFootnotes0/ ~2FootnotesFootnote citations [^N] with source references at the bottom of the page.Add [^N] footnote citationsReferences0/ ~1ReferencesCurated external resources linked via <R> components or cited_by in YAML.Add <R> resource linksQuotes0QuotesSupporting quotes extracted from cited sources to back up page claims.crux citations extract-quotes <id>Accuracy0AccuracyCitations verified against their sources for factual accuracy.crux citations verify <id>
History
Overview
This section traces the development of AI safety as a field, from early theoretical concerns to the current mainstream recognition of AI risks. Understanding this history helps contextualize current debates and institutional structures.
Historical Eras
MIRI Era (2000-2015)HistoricalThe MIRI EraComprehensive chronological account of AI safety's institutional emergence (2000-2015), from MIRI's founding through Bostrom's Superintelligence to mainstream recognition. Covers key organizations,...Quality: 31/100
The field's founding period, dominated by the Machine Intelligence Research InstituteOrganizationMachine Intelligence Research InstituteComprehensive organizational history documenting MIRI's trajectory from pioneering AI safety research (2000-2020) to policy advocacy after acknowledging research failure, with detailed financial da...Quality: 50/100:
Eliezer YudkowskyPersonEliezer YudkowskyComprehensive biographical profile of Eliezer Yudkowsky covering his foundational contributions to AI safety (CEV, early problem formulation, agent foundations) and notably pessimistic views on AI ...Quality: 35/100's early writings on AI risk
Founding of SIAI (later MIRI) in 2000
Development of foundational concepts (orthogonality thesis, instrumental convergenceRiskInstrumental ConvergenceComprehensive review of instrumental convergence theory with extensive empirical evidence from 2024-2025 showing 78% alignment faking rates, 79-97% shutdown resistance in frontier models, and exper...Quality: 64/100)
Superintelligence (2014) brings ideas to academic attention
Deep Learning Era (2015-2022)HistoricalDeep Learning Revolution EraComprehensive timeline documenting 2012-2020 AI capability breakthroughs (AlexNet, AlphaGo, GPT-3) and parallel safety field development, with quantified metrics showing capabilities funding outpac...Quality: 44/100
Deep learning breakthroughs reshape the landscape:
AlphaGo (2016) demonstrates superhuman capability
GPT-2 (2019) shows language model potential
AnthropicOrganizationAnthropicComprehensive reference page on Anthropic covering financials (\$380B valuation, \$19B ARR), safety research (Constitutional AI, mechanistic interpretability, model welfare), governance (LTBT struc...Quality: 74/100 founded (2021) by former OpenAIOrganizationOpenAIComprehensive organizational profile of OpenAI documenting evolution from 2015 non-profit to Public Benefit Corporation, with detailed analysis of governance crisis, 2024-2025 ownership restructuri...Quality: 62/100 safety team
Growing recognition in ML community
FTX/EA Crisis (2022)
The collapse of FTXOrganizationFTXFTX was a major crypto exchange that collapsed in November 2022 due to fraud, with its AI safety relevance stemming from FTX Future Fund grants to organizations like Anthropic and its ties to the e...Quality: 74/100 exposed major fissures in EA-funded AI safety:
FTX collapse and EA's public credibilityE873This article examines the dual impact of the FTX collapse on longtermism: severe reputational and funding damage (including 160M USD in lost Future Fund commitments and FHI's 2024 closure), alongsi...Quality: 50/100 — November 2022 bankruptcy and reputational fallout
EA epistemic failures in the FTX eraE869This page synthesizes post-FTX critiques of EA's epistemic and governance failures, identifying interlocking problems including donor hero-worship, funding concentration in volatile crypto assets, ...Quality: 84/100 — governance, donor vetting, and cultural critiques
EA institutions' responseE870EA institutions responded to the FTX collapse through public condemnation, funding pauses, and community surveys, but were damaged by revelations that warnings about SBF's conduct had been downplay...Quality: 53/100 — community surveys, trust damage, funding gaps
FTX Future FundOrganizationFTX Future FundThe FTX Future Fund was a major longtermist philanthropic initiative that distributed 132M USD in grants (including ~32M USD to AI safety) before collapsing with FTX's November 2022 bankruptcy, exp...Quality: 60/100 — $132M in grants dissolved overnight
Longtermism's credibility after FTXE873This article examines the dual impact of the FTX collapse on longtermism: severe reputational and funding damage (including 160M USD in lost Future Fund commitments and FHI's 2024 closure), alongsi...Quality: 50/100 — philosophical and reputational questions
Early Warnings (2022-2023)HistoricalEarly Warnings EraComprehensive historical overview of AI safety warnings from 1950-2000, documenting foundational thinkers (Turing, Wiener, Good, Vinge) who established core concepts like intelligence explosion, go...Quality: 31/100
AI safety enters public consciousness:
ChatGPT (Nov 2022) captures public attention
Pause letter (March 2023) signed by prominent researchers
Geoffrey HintonPersonGeoffrey HintonComprehensive biographical profile of Geoffrey Hinton documenting his 2023 shift from AI pioneer to safety advocate, estimating 10-20% extinction risk in 5-20 years. Covers his media strategy, poli...Quality: 42/100 leaves Google to speak freely about risks
Congressional hearings on AI safety
Mainstream Era (2023-Present)HistoricalMainstream EraComprehensive timeline of AI safety's transition from niche to mainstream (2020-present), documenting ChatGPT's unprecedented growth (100M users in 2 months), the OpenAI governance crisis, and firs...Quality: 42/100
AI safety becomes a policy priority:
Biden Executive Order on AI (Oct 2023)
Bletchley Park AI Safety Summit (Nov 2023)
AI Safety InstitutesPolicyAI Safety Institutes (AISIs)Analysis of government AI Safety Institutes finding they've achieved rapid institutional growth (UK: 0→100+ staff in 18 months) and secured pre-deployment access to frontier models, but face critic...Quality: 69/100 established globally
Major labs adopt responsible scaling policiesPolicyResponsible Scaling PoliciesComprehensive analysis of Responsible Scaling Policies showing 20 companies with published frameworks as of Dec 2025, with SaferAI grading major policies 1.9-2.2/5 for specificity. Evidence suggest...Quality: 62/100
Key Milestones
Year
Event
Significance
2000
SIAI founded
First AI safety organization
2014
Superintelligence published
Brought ideas to academia
2017
Asilomar Principles
Early multi-stakeholder agreement
2022
FTX collapse
$132M in EA/AI safety funding dissolved; major community reckoning
2022
ChatGPT released
Public awareness breakthrough
2023
UK AI Safety Summit
First major government summit
2024
EU AI ActPolicyEU AI ActComprehensive overview of the EU AI Act's risk-based regulatory framework, particularly its two-tier approach to foundation models that distinguishes between standard and systemic risk AI systems. ...Quality: 55/100 enacted
OpenAIOrganizationOpenAIComprehensive organizational profile of OpenAI documenting evolution from 2015 non-profit to Public Benefit Corporation, with detailed analysis of governance crisis, 2024-2025 ownership restructuri...Quality: 62/100FTX Future FundOrganizationFTX Future FundThe FTX Future Fund was a major longtermist philanthropic initiative that distributed 132M USD in grants (including ~32M USD to AI safety) before collapsing with FTX's November 2022 bankruptcy, exp...Quality: 60/100Machine Intelligence Research InstituteOrganizationMachine Intelligence Research InstituteComprehensive organizational history documenting MIRI's trajectory from pioneering AI safety research (2000-2020) to policy advocacy after acknowledging research failure, with detailed financial da...Quality: 50/100
Policy
Responsible Scaling PoliciesPolicyResponsible Scaling PoliciesComprehensive analysis of Responsible Scaling Policies showing 20 companies with published frameworks as of Dec 2025, with SaferAI grading major policies 1.9-2.2/5 for specificity. Evidence suggest...Quality: 62/100EU AI ActPolicyEU AI ActComprehensive overview of the EU AI Act's risk-based regulatory framework, particularly its two-tier approach to foundation models that distinguishes between standard and systemic risk AI systems. ...Quality: 55/100
Concepts
Ea Institutions Response To The Ftx CollapseEa Institutions Response To The Ftx CollapseEA institutions responded to the FTX collapse through public condemnation, funding pauses, and community surveys, but were damaged by revelations that warnings about SBF's conduct had been downplay...Quality: 53/100Longtermism Credibility After FtxLongtermism Credibility After FtxThis article examines the dual impact of the FTX collapse on longtermism: severe reputational and funding damage (including 160M USD in lost Future Fund commitments and FHI's 2024 closure), alongsi...Quality: 50/100
Historical
Deep Learning Revolution EraHistoricalDeep Learning Revolution EraComprehensive timeline documenting 2012-2020 AI capability breakthroughs (AlexNet, AlphaGo, GPT-3) and parallel safety field development, with quantified metrics showing capabilities funding outpac...Quality: 44/100Mainstream EraHistoricalMainstream EraComprehensive timeline of AI safety's transition from niche to mainstream (2020-present), documenting ChatGPT's unprecedented growth (100M users in 2 months), the OpenAI governance crisis, and firs...Quality: 42/100The MIRI EraHistoricalThe MIRI EraComprehensive chronological account of AI safety's institutional emergence (2000-2015), from MIRI's founding through Bostrom's Superintelligence to mainstream recognition. Covers key organizations,...Quality: 31/100Early Warnings EraHistoricalEarly Warnings EraComprehensive historical overview of AI safety warnings from 1950-2000, documenting foundational thinkers (Turing, Wiener, Good, Vinge) who established core concepts like intelligence explosion, go...Quality: 31/100
Other
Geoffrey HintonPersonGeoffrey HintonComprehensive biographical profile of Geoffrey Hinton documenting his 2023 shift from AI pioneer to safety advocate, estimating 10-20% extinction risk in 5-20 years. Covers his media strategy, poli...Quality: 42/100Eliezer YudkowskyPersonEliezer YudkowskyComprehensive biographical profile of Eliezer Yudkowsky covering his foundational contributions to AI safety (CEV, early problem formulation, agent foundations) and notably pessimistic views on AI ...Quality: 35/100