Bio anchors framework

web

Cold Takes·cold-takes.com/forecasting-transformative-ai-the-biologic...

Credibility Rating

3/5

Good(3)

Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.

Rating inherited from publication venue: Cold Takes

Written by Holden Karnofsky of Open Philanthropy, this post summarizes Ajeya Cotra's influential 'Biological Anchors' report, which has been widely cited in AI safety discussions about timelines and urgency of alignment work.

Metadata

Importance: 72/100blog posteducational

Summary

A layperson-friendly summary of Ajeya Cotra's 'Biological Anchors' framework for forecasting when transformative AI (specifically, AI that can automate all human activities driving scientific progress) might be developed. The method estimates training compute costs by anchoring to the human brain's scale, projecting when such training will become affordable. From nearly all modeled scenarios, it assigns high probability to transformative AI arriving this century.

Key Points

•Introduces 'PASTA' (Process for Automating Scientific and Technological Advancement) as the key threshold for transformative AI.
•The biological anchors method estimates how much compute it would cost to train a brain-scale AI model and projects when that cost becomes feasible.
•The framework generates a wide range of estimates from aggressive to conservative, but nearly all place transformative AI as likely this century.
•Author notes the method's complexity and limited validation history as weaknesses compared to simpler forecasting approaches.
•Serves as an accessible entry point to Ajeya Cotra's full technical report on AI timelines.

Cited by 1 page

Page	Type	Quality
Holden Karnofsky	Person	40.0

Cached Content Preview

HTTP 200Fetched Apr 9, 202632 KB

Forecasting transformative AI: the "biological anchors" method in a nutshell 
 
 
 
 
 -->
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 -->
 -->

 
 -->
 
 
 -->
 

 

 

 

 

 

 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 

 
 

 Subscribe (free) 

 
 
 
 
 
 
 
 
 Audio also available by searching Stitcher, Spotify, Google Podcasts, etc. for "Cold Takes Audio" 

 

-->

 
 
 
 Today&#x2019;s world 
 
 
 
 
 
 
 Transformative AI 
 
 
 
 
 
 
 
 
 Digital people 
 World of 
 
 
 
 Misaligned AI 
 World run by 
 Something else 
 
 
 or 
 or 
 
 
 Stable, galaxy-wide&#10; civilization 
 
 
 
 
 
 
 

 
 This is one of 4 posts summarizing hundreds of pages of technical reports focused almost entirely on forecasting one number: the year by which transformative AI will be developed. 1 

 
By "transformative AI," I mean "AI powerful enough to bring us into a new, qualitatively different future." I specifically focus on what I'm calling PASTA : AI systems that can essentially automate all of the human activities needed to speed up scientific and technological advancement.

 
The sooner PASTA might be developed, the sooner the world could change radically , and the more important it seems to be thinking today about how to make that change go well vs. poorly. 

 
This post is a layperson-compatible summary of Ajeya Cotra's "Forecasting Transformative AI with Biological Anchors " (which I'll abbreviate below as "Bio Anchors" ), and its pros and cons. 2 It is the forecast I find most informative for transformative AI, with some caveats:

 
 This approach is relatively complex, and it requires a fairly large number of assumptions and uncertain estimates. These qualities make it relatively difficult to explain, and they are also a mark against the method's reliability. 

 Hence, as of today, I don't think this method is as trustworthy as the examples I gave previously for forecasting a qualitatively different future. It does not have the simplicity and directness of some of those examples, such as modeling COVID-19's spread. And while climate modeling is also very complex, climate modeling has been worked on by a large number of experts over decades, whereas the Bio Anchors methodology doesn't have much history.

 
 
Nonetheless, I think it is the best available "best guess estimate" methodology for transformative AI timelines as of today. And as discussed in the final section , one can step back from a lot of the details to see that this century will likely see us hit some of the more "extreme" milestones in the report that strongly suggest the feasibility of transformative AI. 

 (Note: I've also written up a follow-up post about this framework for skeptical readers. See “Biological anchors” is about bounding, not pinpointing, AI timelines .)

 
The basic idea is:

 

 Modern AI models c

... (truncated, 32 KB total)

Resource ID: 62d29d310d596d2a | Stable ID: sid_qBcoYPahh9