Back
Compute trend analysis
webCredibility Rating
4/5
High(4)High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: Epoch AI
Epoch AI empirical analysis useful for understanding the economics of frontier AI development and informing governance discussions about compute thresholds and access concentration.
Metadata
Importance: 62/100blog postanalysis
Summary
This Epoch AI analysis tracks historical trends in the monetary cost of training machine learning systems, examining how dollar costs have evolved alongside compute scaling. It provides empirical data on training cost trajectories to inform forecasts about future AI development economics and accessibility.
Key Points
- •Training costs for frontier ML models have grown dramatically over time, with top models now requiring billions of dollars in compute expenditure.
- •Despite hardware efficiency improvements reducing cost-per-FLOP, total training costs have risen due to exponentially increasing model scale.
- •Cost trends have implications for who can develop frontier AI systems, concentrating capabilities among well-resourced actors.
- •Historical data enables forecasting of future training costs, relevant for governance and safety planning timelines.
- •Declining per-unit compute costs mean capable models become more accessible over time, raising proliferation concerns.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| AI Proliferation Risk Model | Analysis | 65.0 |
Cached Content Preview
HTTP 200Fetched Apr 9, 202698 KB
Trends in the Dollar Training Cost of Machine Learning Systems | Epoch AI
Oct
NOV
Dec
08
2023
2024
2025
success
fail
About this capture
COLLECTED BY
Organization: National Archives and Records Administration
National Archives and Records Administration crawl performed by Internet Archive. This data is currently not publicly accessible.
Collection: NARA 118th Congressional Crawl
This crawl of online resources of the 118th U.S. Congress was performed by Internet Archive on behalf of the United States National Archives & Records Administration (NARA).
TIMESTAMPS
The Wayback Machine - http://web.archive.org/web/20241108001435/https://epochai.org/blog/trends-in-the-dollar-training-cost-of-machine-learning-systems
Articles
Research
Trends
Data
Team
Careers
Support us
Menu
Close
Articles
Research
Trends
Data
Team
Careers
Support us
Search
Search epochai.org
Search
Enter a query to search for results
Placeholder
Articles
Trends in the Dollar Training Cost of Machine Learning Systems
report
Trends in the Dollar Training Cost of Machine Learning Systems
I combine training compute and GPU price-performance data to estimate the cost of compute in US dollars for the final training run of 124 machine learning systems published between 2009 and 2022, and find that the cost has grown by approximately 0.5 orders of magnitude per year.
Published
Jan 31, 2023
Authors
Ben Cottier
Resources
Dataset
Source Code
Cite
Contents
Summary
Why study dollar training costs?
Method
Background on methods to estimate the dollar cost of training compute
Estimating training cost from training compute and GPU price-performance
Method 1: Using the overall GPU price-performance trend
Method 2: Using the price-performance of actual hardware used to train ML systems
Dataset
Code
Large-scale systems
Results
Method 1: Using the overall GPU price-performance trend for all ML systems (n=124)
Growth rate of training cost for all ML systems: 0.51 OOMs/year
Growth rate of training cost for large-scale ML systems: 0.2 OOMs/year
Method 2: Using the price-performance
... (truncated, 98 KB total)Resource ID:
dff8fae99b47e61d | Stable ID: YTQ5YmRiYz