Research foundation with $15M funding (from Macroscopic Ventures) studying how AI systems can learn to cooperate and how to design environments incentivizing cooperation. Co-founded by Allan Dafoe (now Director of Frontier Safety and Governance at Google DeepMind), Gillian Hadfield, and others. Published foundational "Open Problems in Cooperative AI" (2020). Key focus areas include multi-agent cooperation, preventing destructive AI competition, and human-AI coordination.
Multi-Agent SafetyApproachMulti-Agent SafetyMulti-agent safety addresses coordination failures, conflict, and collusion risks when AI systems interact. A 2025 report from 50+ researchers identifies seven key risk factors; empirical studies s...Quality: 68/100
Analysis
International AI Coordination Game ModelAnalysisInternational AI Coordination Game ModelGame-theoretic analysis demonstrating that US-China AI coordination defaults to mutual defection (racing) because defection dominates when cooperation probability falls below 50%, with current esti...Quality: 59/100CICEROProjectCICEROMeta AI agent achieving human-level performance in Diplomacy — a game requiring negotiation, alliance formation, and multi-party coordination. Published in Science (2022). Demonstrated AI can learn...OpenSpielProjectOpenSpielMost widely used framework for game-theoretic reinforcement learning research. Open-source library by Google DeepMind (Marc Lanctot et al.). Supports extensive-form games, normal-form games, and mu...AI Safety Culture Equilibrium ModelAnalysisAI Safety Culture Equilibrium ModelGame-theoretic model identifying three equilibria for AI lab safety culture: racing-dominant (current state, S=0.25), safety-competitive (S>0.6), and regulation-imposed (S=0.15-0.25). Key finding: ...Quality: 65/100Racing Dynamics Impact ModelAnalysisRacing Dynamics Impact ModelThis model quantifies how competitive pressure between AI labs reduces safety investment by 30-60% compared to coordinated scenarios and increases alignment failure probability by 2-5x through pris...Quality: 61/100Multipolar Trap Dynamics ModelAnalysisMultipolar Trap Dynamics ModelGame-theoretic analysis of AI competition traps showing universal cooperation probability drops from 81% (2 actors) to 21% (15 actors), with 5-10% catastrophic lock-in risk and 20-35% partial coord...Quality: 61/100
Other
Jesse CliftonPersonJesse CliftonAI safety researcher focused on multi-agent safety, cooperative AI, and game-theoretic approaches to alignment. Previously at the Center for AI Safety.Noam BrownPersonNoam BrownResearcher at OpenAI (formerly Meta AI). Built CICERO — AI achieving human-level Diplomacy (published in Science 2022), requiring negotiation, alliance formation, and multi-party coordination. Also...Yoshua BengioPersonYoshua BengioComprehensive biographical overview of Yoshua Bengio's transition from deep learning pioneer (Turing Award 2018) to AI safety advocate, documenting his 2020 pivot at Mila toward safety research, co...Quality: 39/100
Organizations
US AI Safety InstituteOrganizationUS AI Safety InstituteThe US AI Safety Institute (AISI), established November 2023 within NIST with $10M budget (FY2025 request $82.7M), conducted pre-deployment evaluations of frontier models through MOUs with OpenAI a...Quality: 91/100Olas (Autonolas)OrganizationOlas (Autonolas)Protocol for building and coordinating autonomous multi-agent systems across blockchains. Uses Tendermint consensus so multiple agents must agree before executing on-chain. Decentralized marketplac...
Concepts
Cooperate-BotConceptCooperate-BotDesign analysis of a 'cooperate-bot' — an AI agent given a recurring personal budget to handle reciprocity, public goods contributions, and professional relationship maintenance. Maps the automatio...Quality: 50/100Heavy Scaffolding / Agentic SystemsConceptHeavy Scaffolding / Agentic SystemsComprehensive analysis of multi-agent AI systems with extensive benchmarking data showing rapid capability growth (77.2% SWE-bench, 5.5x improvement 2023-2025) but persistent reliability challenges...Quality: 57/100