Back
Superforecaster Predictions of Long-Term Impacts
webThis Manifund project funds superforecasters to predict long-term RCT treatment effects, generating empirical evidence on forecast accuracy decay over time and forecaster vs. expert performance—relevant to AI safety debates about the tractability of long-term impact prediction and longtermist decision-making.
Metadata
Importance: 38/100otherprimary source
Summary
This project pays 30 superforecasters to forecast 5-10 year treatment effects from randomized controlled trials, enabling rapid resolution of long-run forecasts. It investigates how forecast accuracy decays with time horizon and when superforecasters outperform domain experts, providing empirical grounding for debates about the predictability of long-term impacts relevant to longtermism.
Key Points
- •Pays 30 superforecasters (from GJO, Metaculus, Manifold) to forecast long-run RCT treatment effects across 7 trials
- •Addresses how forecast accuracy decays with time horizon—critical for evaluating longtermist expected value arguments
- •Compares superforecaster performance to domain experts in areas beyond geopolitics (e.g., economics)
- •Uses RCT structure to resolve long-run forecasts quickly while maintaining genuine long-horizon prediction challenge
- •Contributes to understudied 'impact/causal forecasting' vs. more common 'state forecasting' research
Cached Content Preview
HTTP 200Fetched Apr 11, 202615 KB
Superforecaster predictions of long-term impacts | Manifund
Nov
DEC
Jan
30
2024
2025
2026
success
fail
About this capture
COLLECTED BY
Collection: Save Page Now Outlinks
TIMESTAMPS
The Wayback Machine - https://web.archive.org/web/20251230135708/https://manifund.org/projects/superforecaster-predictions-of-long-term-impacts
Manifund
Home
Login
About
People
Categories
Newsletter
Home
About
People
Categories
Login
Create
2
Superforecaster predictions of long-term impacts
ACX mini-grants
Forecasting
David Rhys Bernard
Complete
Impact certificate
$2,999raised
$2,000valuation
Sign in to trade
p]:prose-li:my-0 text-gray-900 prose-blockquote:text-gray-600 prose-a:font-light prose-blockquote:font-light font-light break-anywhere empty:prose-p:after:content-["\00a0"]">
Project description
I will pay 30 forecasters with excellent track records on Good Judgement Open, Metaculus and Manifold (superforecasters) to make forecasts of long-run treatment effects from randomised controlled trials. This will allow me to provide new evidence on two critical questions in the forecasting space (1) how does forecast accuracy decay with time horizon?, and (2) when are (super)forecasters better than domain experts?
Why forecasts of long-term treatment effects from randomised controlled trials (RCTs)? Firstly, most research on forecasting is about ‘state’ forecasts, what the world will look like in the future. More relevant for those seeking to improve the world are ‘impact’ (or causal) forecasts, the difference between what would happen if we take action X and what would happen if we did not take action X. The treatment effects of RCTs are causal impacts and by collecting forecasts of them I contribute to this understudied area of forecasting.
Secondly, using RCTs allows us to resolve long-run forecasts more quickly. I will collect forecasts for the 5-10 year results from 7 different RCTs. These RCTs are already underway and the long-run results will be available to me in spring 2023 so I will be able to resolve the long-run forecasts soon. However, the only information that is available about the RCTs is a short-run set of results, typically observed 2 years after each RCT started. As such, if the long-run results are from year 10, the long-run forecast of these results approximates an 8-year forecast but resolves much more quickly. It is not possible for the forecasters to know anything about what happened in each RCT between years 2 and 10, so the forecast is a real long-run forecast.
Why care about question (1) how does forecast accuracy decay with time horizon? Firstly, it’s important to know how much we can trust long-range forecasts in a variety of domains when we’re making policies and decisions with long-run impacts. Secondly, a common objection to longtermi
... (truncated, 15 KB total)Resource ID:
64bc0b68147c2253 | Stable ID: sid_leQmQPlybZ