Projects
AI safety tools, platforms, forecasting systems, and research projects.
Projects
62
With Website
35
With Organization
38
Loading…
AI safety tools, platforms, forecasting systems, and research projects.
| Description | Coverage | Links | |||
|---|---|---|---|---|---|
| AAA-ICDR AI Arbitrator | First AI-native arbitrator from a major arbitration institution (AAA-ICDR). Opt-in for two-party, documents-only construction disputes. Trained on 1,500+ real cases in collaboration with McKinsey's QuantumBlack. Human arbitrators oversee, revise, and authorize each outcome. Launched November 3, 2025, expanding to additional industries. | active | |||
| AI Economist | Salesforce Research reinforcement learning framework that simultaneously optimizes tax policy while agents learn economic behavior. Published in Science Advances (2022). Showed RL-designed tax policies improved equality-productivity tradeoffs over US tax schedules. | archived | |||
| AI Forecasting Benchmark Tournament | Quarterly competition run by Metaculus comparing human Pro Forecasters against AI forecasting bots. | Metaculus | active | ||
| AI Safety Papers | A UI on top of an existing database about AI Safety Papers. The underlying database is not being updated, so the data is fairly old (2020). | QURI (Quantified Uncertainty Research Institute) | archived | ||
| AI Watch | AI Watch is a tracking database by Issa Rice that monitors AI safety organizations, people, funding, and publications as part of his broader knowledge infrastructure ecosystem. The article provides useful context about Rice's systematic approach to documentation but lacks concrete details about AI Watch's actual scope, methodology, or current operational status. | active | |||
| All Our Ideas | Free, open-source wiki survey tool from Princeton. Uses pairwise comparisons — participants repeatedly choose between two options and the system ranks ideas. NYC used it for PlaNYC 2030 (1,436 respondents, 464 ideas). | active | |||
| Caliptra | Open-source silicon Root of Trust (RoT) security subsystem designed for integration into SoCs. Provides identity, measured boot, and attestation capabilities. Founded by Microsoft, AMD, Google, and NVIDIA in 2022 under the CHIPS Alliance. Version 2.1 adds quantum-resilient cryptography (ML-DSA for post-quantum signatures, ML-KEM for key exchange). ~1.6M logic gates with dual RISC-V cores. Potentially the most promising cross-vendor foundation for hardware-enabled AI governance. | active | |||
| Case Law for AI | AIDF project exploring judicial-inspired approaches for connecting deliberative public input into AI alignment and inference systems. Uses case law methods to create knowledge representation that preserves deliberative nuance. | AI & Democracy Foundation | active | ||
| CICERO | Meta AI agent achieving human-level performance in Diplomacy — a game requiring negotiation, alliance formation, and multi-party coordination. Published in Science (2022). Demonstrated AI can learn complex multi-party coordination with humans. | archived | |||
| Climate TRACE Platform | AI + satellite platform independently tracking global greenhouse gas emissions. 352M+ assets tracked. Used at COP summits. Reduces information asymmetry that is a key barrier to climate collective action. | Climate TRACE | active | ||
| CONSUL Platform | Open-source government participation platform for proposals, debates, voting, participatory budgets, and collaborative legislation. ~200 governments in 39 countries. | CONSUL Democracy | active | ||
| Decidim Platform | Open-source participatory democracy platform. Supports participatory budgeting, citizen proposals, and deliberation. 500+ institutions, 3M+ users worldwide. | Decidim Association | active | ||
| Deliberation.io | Open-source platform from Stanford HAI's Digital Economy Lab and MIT GOV/LAB using Socratic dialogue for large-scale AI-assisted deliberation. Four components: individual engagement, AI agent interaction, "Face of the Crowd" opinion distribution, "Dynamic Deliberation." First US city deployment: Washington DC (July 2025). | active | |||
| Democracy Levels Framework | Framework by Aviv Ovadya and 13 co-authors for evaluating the degree to which decisions in a given domain are made democratically. Provides milestones toward meaningfully democratic AI. Published November 2024 (arxiv 2411.09222). | AI & Democracy Foundation | active | ||
| DemocracyOS | Open-source platform for online deliberation. Citizens vote on, track, and debate legislation. Built by Democracia en Red (Argentina). Translated into 15 languages. Used in Tunisia (constitutional debate), Mexico, Kenya. | maintained | |||
| Donations List Website | Comprehensive documentation of an open-source database tracking $72.8B in philanthropic donations (1969-2023) across 75+ donors, with particular coverage of EA/AI safety funding. The page thoroughly describes the tool's features, data coverage, and limitations, but is purely descriptive reference material about a data tool rather than analysis of AI safety funding patterns. | active | |||
| ElizaOS | Open-source TypeScript framework for creating autonomous on-chain AI agents. Originally the ai16z DAO's AI-managed venture fund. Rebranded January 2025. Features include LLM integration, long-term memory, and multi-chain platform connectors. 2,200+ agents built. Building toward "Multi-Agent Swarms" where specialized agents coordinate. | Eliza Labs | active | ||
| FlexHEG (Flexible Hardware-Enabled Guarantees) | Research project led by Yoshua Bengio developing open-source hardware-enabled guarantees for AI governance. Proposes a "Guarantee Processor" that monitors accelerator usage and verifies compliance, combined with a "Secure Enclosure" providing physical tamper protection. Funded with $4.1M from the Survival and Flourishing Fund (2024). Published interim report (September 2024), technical options paper (arXiv:2506.03409), and international security applications paper (arXiv:2506.15100). Interlock-based design gives the Guarantee Processor direct access to the accelerator data path. Estimated 3.7-7.9 years for integrated hardware deployment. | active | |||
| ForecastBench | Dynamic, contamination-free benchmark for evaluating LLM forecasting capabilities, published at ICLR 2025. | Forecasting Research Institute (FRI) | active | ||
| Foretold.io | A prediction platform meant for large sets of judgmental predictions. Ozzie Gooen worked on this for much of 2018-2020. Later work halted to favor smaller applications and research. Currently not functioning. | QURI (Quantified Uncertainty Research Institute) | abandoned | ||
| GeoMatch (Refugee Resettlement) | AI algorithm predicting refugees' expected employment success by location, with placement recommendation tool. Deployed or piloted in US, Netherlands, Norway, Switzerland. Built by Stanford/ETH Zurich Immigration Policy Lab. | active | |||
| Gitcoin Grants (Quadratic Funding) | Largest practical implementation of quadratic funding. Number of contributors matters more than amount, mathematically optimizing public goods provision. $67M+ distributed to 5,000+ open-source projects. Now using Dedicated Domain Allocation (DDA) model. | Gitcoin | active | ||
| Gitcoin Passport | Decentralized identity and Sybil defense. Aggregates "stamps" (credentials) from web2/web3 authenticators, calculates forgery cost to produce a credibility score. | Gitcoin | active | ||
| Grokipedia | xAI's AI-generated encyclopedia launched October 2025, growing to 6M+ articles with documented quality concerns including political bias and scientific inaccuracies. | xAI | active | ||
| Guesstimate | Web-based spreadsheet for making estimates with probability distributions. Precursor to Squiggle. Originally created by Ozzie Gooen in 2016. Features a library of 17,000+ community models. Still maintained but active development shifted to Squiggle and Squiggle Hub. | Ozzie Gooen | maintained | ||
| Habermas Machine | Google DeepMind's LLM-based "caucus mediator" trained to generate group opinion statements identifying areas of agreement on complex political/social issues. Published in Science (October 2024). Tested on 5,700+ UK participants. AI-mediated statements consistently preferred over human-written ones for quality, clarity, and perceived fairness. | Google DeepMind | active | ||
| Kayrros Methane Watch | Global methane monitoring platform using Sentinel-5P satellite + AI for daily emissions detection. Launched "methane GPT" LLM for public querying. TIME Best Inventions 2025. | Kayrros | active | ||
| Kialo | Structured debate and argument mapping platform. Users build pro/con argument trees on topics. Used in education and for public deliberation. | active | |||
| Kleros Court | Decentralized arbitration system. Jurors stake tokens, are randomly selected, and rewarded for voting with the majority (Schelling point mechanism). V2 deployed on Arbitrum One (November 2024). Exploring AI agents as jurors. | Kleros | active | ||
| Learned Hand | AI tool that distills legal motions, learns individual judges' writing styles, and drafts tentative rulings. "Deep Verify" fact-checks every sentence against case law. Piloting in LA Superior Court, deployed in 10 US states. Judges not required to disclose AI use. | active | |||
| LiquidFeedback | Free software for liquid democracy — vote delegation, initiative improvement, and collective decision-making. Used primarily by Pirate Parties in Germany, Italy, Austria, Norway, France, Netherlands. | active | |||
| Longterm Wiki | A self-referential documentation page describing the Longterm Wiki platform itself—a strategic intelligence tool with ~550 pages, crux mapping of ~50 uncertainties, and quality scoring across 6 dimensions. Features include entity cross-linking, interactive causal diagrams, and structured YAML database layer for structured entity data. | QURI (Quantified Uncertainty Research Institute) | active | ||
| Loomio | Decision-making platform for groups. Supports proposals, polls, and structured discussion. Used by governments and cooperatives worldwide. | Loomio Cooperative | active | ||
| MetaDAO Futarchy Protocol | First functional futarchy governance on mainnet. Participants trade on prediction markets about whether proposals will increase token value, replacing traditional voting. Generates ~$4,825/day in revenue. 11 projects using "Futarchy as a Service." | MetaDAO | active | ||
| Metaforecast | Forecast aggregation platform combining predictions from 10+ sources into a unified search interface. | QURI (Quantified Uncertainty Research Institute) | abandoned | ||
| MIT AI Risk Repository | The MIT AI Risk Repository catalogs 1,700+ AI risks from 65+ frameworks into a searchable database with dual taxonomies (causal and domain-based). Updated quarterly since August 2024, it provides the first comprehensive public catalog of AI risks but is limited by framework extraction methodology and incomplete coverage of some risk domains. | Massachusetts Institute of Technology | active | ||
| OpenSpiel | Most widely used framework for game-theoretic reinforcement learning research. Open-source library by Google DeepMind (Marc Lanctot et al.). Supports extensive-form games, normal-form games, and multi-agent RL environments. | active | |||
| Optimism Retroactive Public Goods Funding | Rewards contributors after the fact for demonstrated impact. 60M+ OP tokens distributed across multiple rounds. Novel mechanism: fund what already worked rather than predicting what will. | active | |||
| Org Watch | Org Watch is a tracking website by Issa Rice that monitors EA and AI safety organizations, but the article lacks concrete information about its actual features, scope, or current status. The piece reads more like speculative analysis about what the tool might do rather than documentation of an established, operational product. | active | |||
| Pactum AI Negotiation Platform | Autonomous AI agents conducting full negotiation conversations with suppliers. Walmart: 3% average gain, 35-day payment term extension. Negotiation cycles reduced from weeks to days (individual negotiations can complete in ~15 minutes). 2-30% value generation on negotiated spend. | Pactum AI | active | ||
| Polis | Open-source AI-assisted deliberation platform using ML clustering to map opinion landscapes. Participants submit brief statements and vote agree/disagree/pass; algorithms cluster similar voters and surface bridging consensus. No reply button (eliminates trolling). Deployed in dozens of countries across five continents. vTaiwan: 26+ issues, 80% implementation rate. Anthropic CCAI: ~1,000 participants contributed to training Claude's constitution. | Computational Democracy Project | active | ||
| Proof of Humanity | Sybil-resistant registry using video verification and vouching to prove human uniqueness. Issues soulbound-style credentials. Built by Kleros. | Kleros | active | ||
| RoastMyPost | LLM-powered tool for evaluating blog posts, research papers, and other written content. Runs multiple AI evaluator personas (skeptic, methodologist, clarity checker, etc.) and produces structured feedback with scores. Free and open-source. Launched December 2025. | QURI (Quantified Uncertainty Research Institute) | active | ||
| Snapshot Voting | Gasless off-chain voting platform. Votes stored as signed messages on IPFS. The dominant DAO voting tool — 96% of major DAO votes, 30,000+ projects. | Snapshot Labs | active | ||
| Spliddit | Fair division platform by Ariel Procaccia. Provides provably fair algorithms for dividing rent, goods, chores, and credit among groups. Based on computational social choice theory. | maintained | |||
| Squiggle | Domain-specific programming language for probabilistic estimation with native distribution types and Monte Carlo sampling. | QURI (Quantified Uncertainty Research Institute) | active | ||
| Squiggle Hub | Web platform for creating, sharing, and discovering Squiggle models. Features include model groups, version history, AI-assisted editing via SquiggleAI, and a growing library of community-contributed probabilistic models. Successor to Guesstimate's model sharing functionality. | QURI (Quantified Uncertainty Research Institute) | active | ||
| SquiggleAI | LLM-powered tool for generating probabilistic models in Squiggle from natural language descriptions. | QURI (Quantified Uncertainty Research Institute) | active | ||
| Stampy / AISafety.info | AISafety.info is a volunteer-maintained wiki with 280+ answers on AI existential risk, complemented by Stampy, an LLM chatbot searching 10K-100K alignment documents via RAG. Features include a Discord bot bridging YouTube comments, PageRank-style karma voting for answer quality control, and the Distillation of AI safety arguments into accessible formats. | active | |||
| Talk to the City | Open-source LLM tool for scaling deliberation. Analyzes large-scale public input (surveys, interviews, social media), clusters opinions using semantic analysis, and creates interactive navigable opinion maps. Deployed in Taiwan (moda ministry), Chatham House "Recursive Public" project (1,000 participants), Tokyo election polling, UNFCCC climate negotiations. Audrey Tang says it reduces "broad-listening" cost to "essentially zero." | AI Objectives Institute | active | ||
| ThinkScape | Conversational Swarm Intelligence platform by Unanimous AI. Divides large groups into subgroups woven together by real-time AI "Conversational Surrogates." Enables real-time deliberation among groups of unlimited size. Used at UNFCCC Bonn climate negotiations (2025). CMU study: groups averaged effective IQ 128. | Unanimous AI | active | ||
| Timelines Wiki | Timelines Wiki is a specialized MediaWiki project documenting chronological histories of AI safety and EA organizations, created by Issa Rice with funding from Vipul Naik in 2017. While useful as a historical reference source, it primarily serves as documentation infrastructure rather than providing original analytical insight. | active | |||
| Truth Terminal | Autonomous AI agent operating a Twitter/X account, generating content and accumulating crypto wealth. Created the GOAT memecoin phenomenon (peaked above $600M market cap). First AI agent to become a crypto millionaire. Created by Andy Ayrey; received $50K BTC from Marc Andreessen. | active | |||
| UMA Optimistic Oracle | Dispute resolution mechanism for prediction markets. Assertions assumed true unless challenged. Now using LLMs to propose and dispute data. >99% accuracy. Primary resolution layer for Polymarket. | UMA Protocol | active | ||
| Utility Function Extractor | A tool to elicit comparisons from people and convert those into a simple utility function. Used privately for occasional experiments. | QURI (Quantified Uncertainty Research Institute) | archived | ||
| Waves | Largest trial of digital democracy in Britain. AI-powered consensus-finding on local government issues. EUR 1M project. Camden Council first deployment (Sept 2025), South Staffordshire (Spring 2026), expansion planned. | Demos | active | ||
| Wikipedia Views | This article provides a comprehensive overview of Wikipedia pageview analytics tools and their declining traffic due to AI summaries reducing direct visits. While well-documented, it's primarily about web analytics infrastructure rather than core AI safety concerns. | active | |||
| X Community Notes | Community Notes uses a bridging algorithm requiring cross-partisan consensus to display fact-checks, reducing retweets 25-50% when notes appear. However, only 8.3% of notes achieve visibility, taking median 7 hours (mean 38.5 hours) by which time 96.7% of spread has occurred, limiting aggregate effectiveness to approximately 3-4% of misinformation reduction. | active | |||
| x402 Payment Protocol | Coinbase-developed open standard embedding stablecoin micropayments into HTTP. Enables AI agents to autonomously pay for services without accounts. x402 Foundation co-founded by Coinbase and Cloudflare; Google and Visa joined. ~$28K daily volume (March 2026). | active | |||
| XPT (Existential Risk Persuasion Tournament) | Four-month structured forecasting tournament bringing together superforecasters and domain experts through adversarial collaboration. | Forecasting Research Institute (FRI) | active | ||
| Your Priorities | Open-source idea generation and participatory budgeting platform by Citizens Foundation. Rated #1 on People Powered 2025 Participation Platform Ratings. Better Reykjavik engaged 70,000 of 120,000 residents with 10,000+ ideas. | Citizens Foundation | active | ||
| Zodiac | Modular governance expansion pack for DAOs by Gnosis Guild. Composable modules include Governor (token voting), Reality (off-chain signals trigger on-chain), and Delay (time-locked execution). | Gnosis Guild | active |