Back
The Superforecasters Track Record - Good Judgment Inc.
webCredibility Rating
3/5
Good(3)Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.
Rating inherited from publication venue: Good Judgment
Relevant to AI safety for understanding how structured forecasting methods can be applied to anticipate AI-related risks and policy outcomes; Good Judgment's superforecasters are sometimes consulted on AI timelines and governance questions.
Metadata
Importance: 42/100organizational reportreference
Summary
Good Judgment Inc. documents the empirical track record of its superforecasters—elite forecasters identified through the Good Judgment Project—showing their consistent outperformance of intelligence analysts, prediction markets, and general public forecasters. The page highlights calibration, accuracy metrics, and real-world forecasting achievements as evidence of the value of structured probabilistic forecasting.
Key Points
- •Superforecasters outperform CIA analysts with access to classified information by roughly 30% on geopolitical forecasting questions.
- •Performance is measured using Brier scores, providing a rigorous quantitative basis for comparing forecasting accuracy.
- •The superforecaster methodology emphasizes calibration, active updating, and aggregation of diverse viewpoints.
- •Track record spans domains including geopolitics, economics, and public health, suggesting broad applicability.
- •Demonstrates that systematic training and selection can produce reliably superior probabilistic forecasters.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Good Judgment (Forecasting) | Organization | 50.0 |
Cached Content Preview
HTTP 200Fetched Apr 9, 20265 KB
The Superforecasters’ Track Record - Good Judgment
Resources > The Superforecasters’ Track Record
How accurate are the Superforecasters?
Superforecasters represented the cream of the crop of the Good Judgment Project forecasters. And they’ve proven themselves time and time again since turning professional in 2015. Below, we present data about their track record in both absolute and relative terms.
Superforecasters have beaten all head-to-head competitors
Geopolitical forecasters
Superforecasters beat all competing research teams in the IARPA ACE tournament by 35-72%.
Learn more »
US intelligence analysts
Good Judgment was over 30% more accurate than intelligence analysts with access to classified information.
Learn more »
ClearerThinking.org competition
Superforecasters outperformed both client experts & a crowd-wisdom group in forecasting the new administration's policies.
Learn more »
Hybrid human-machine systems
100 Superforecasters defeated hybrid systems combining machine learning with crowd forecasts from 1,000+ people.
Learn more »
“Team Good Judgment, led by Philip Tetlock and Barbara Mellers of the University of Pennsylvania, beat the control group by more than 50%. This is the largest improvement in judgmental forecasting accuracy observed in the literature.”
Steven Rieber, Program Manager, IARPA
The Superforecasters’ Track Record: Interest Rates, Covid-19, and the Russian Invasion of Ukraine
A post-mortem exercise is a key step in the Superforecasting process. For every forecast we make, we keep score so we can learn and improve. As an example, we compare the Superforecasters’ track record in forecasting inflection points in the Fed’s policy with futures markets. (Superforecasters were 66% more accurate than the futures.) We look back at Superforecasters’ performance in forecasting the reopening in the early stages of the Covid-19 pandemic. (Superforecasters offered valuable early signal where others failed.) Because we take accountability in forecasting seriously, we’re also sharing our post-mortem for the question whether Russia would invade Ukraine and present lessons learned.
“Superforecasting the Fed’s Target Range,” C. Karvetski (2023)
Download
now »
Post-Mortem: Lessons Learned in Superforecasting the Russian Invasion of Ukraine
Download
now »
COVID-19: Superforecasting the Reopening
Download
now »
“Superforecasters: A Decade of Stochastic Dominance,” C. Karvetski (2021)
Download
now »
See the future sooner
An analysis of Good Judgment Project forecasts by UC-Irvine decision scientist Mark Steyvers found that Superforecasters anticipa
... (truncated, 5 KB total)Resource ID:
9b4cdbd059586751 | Stable ID: sid_P9qPBzhejZ