Back
METR (Model Evaluation & Threat Research) - About
webCredibility Rating
4/5
High(4)High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: METR
METR (formerly ARC Evals) is a key organization in the AI safety ecosystem, providing evaluations that major labs use in their safety commitments; their ARA evaluations are referenced in multiple responsible scaling policies.
Metadata
Importance: 62/100homepage
Summary
METR is an organization focused on evaluating AI models for dangerous capabilities, particularly autonomous replication and adaptation (ARA) risks. They develop evaluation frameworks and conduct assessments to determine whether frontier AI systems pose catastrophic risks before deployment. Their work informs AI safety policy and responsible scaling decisions at major AI labs.
Key Points
- •METR develops standardized evaluations for dangerous AI capabilities, especially autonomous replication and adaptation (ARA) abilities
- •The organization conducts third-party model evaluations for frontier AI labs to inform deployment and safety decisions
- •METR's threat research focuses on scenarios where AI could pose catastrophic or existential risks to humanity
- •Their evaluation work feeds into responsible scaling policies at labs like Anthropic and Google DeepMind
- •METR operates as an independent safety-focused organization to provide credible external oversight of frontier models
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| METR | Organization | 66.0 |
Cached Content Preview
HTTP 200Fetched Apr 9, 202610 KB
About METR
Research
Notes
Updates
About
Donate
Careers
Search
-->
Research
Notes
Updates
About
Donate
Careers
Menu
About METR
Model Evaluation and Threat Research
What we do
METR (pronounced 'meter') evaluates frontier AI models to help companies and wider society understand AI capabilities and what risks they pose.
Most of our research consists of evaluations assessing the extent to which an AI system can autonomously carry out substantial tasks , including general-purpose tasks like conducting research or developing an app, and concerning capabilities such as conducting cyberattacks or making itself hard to shut down. Recently, we've begun studying the effects of AI on real-world software developer productivity as well as potential AI behavior that threatens the integrity of evaluations and mitigations for such behavior .
Examples of our evaluation research:
Measuring AI Ability to Complete Long Tasks
We found that the length of tasks AI agents can complete has doubled approximately every 7 months for 6 years. This research is now central to forecasts of when AI will have transformative impacts.
GPT-5.1-Codex-Max Evaluation
We evaluated GPT-5.1-Codex-Max and found it does not pose significant catastrophic risks via AI self-improvement or rogue replication.
Measuring the Impact of AI on Open-Source Developer Productivity
We conducted an RCT and found that experienced open-source developers systematically overestimate how beneficial AI tools are for their productivity.
METR also prototypes governance approaches which use AI systems' measured or forecasted capabilities to determine when better risk mitigations are needed for further scaling. This included prototyping the Responsible Scaling Policies approach, which has been adopted by nine leading AI developers .
Our mission
METR’s mission is to develop scientific methods to assess catastrophic risks stemming from AI systems’ autonomous capabilities and enable good decision-making about their development.
At some point, AI systems will probably be able to do most of what humans can do, including developing new technologies; starting businesses and making money; finding new cybersecurity exploits and fixes; and more. This could change the world quickly and drastically, with potential for both enormous good and enormous harm. Unfortunately, it’s hard to predict exactly when and how this might happen. Being able to measure the autonomous capabilities of AI systems will allow companies and policymakers to see whe
... (truncated, 10 KB total)Resource ID:
9ece1a3a9a30d8c1 | Stable ID: sid_sYUj0h8sm9