Back
Coherent extrapolated volition - Wikipedia
referenceCredibility Rating
3/5
Good(3)Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.
Rating inherited from publication venue: Wikipedia
A Wikipedia overview of a foundational AI alignment concept; note the article contains an LLM-generated content warning and may include unverified claims, so primary sources (Yudkowsky's 2004 paper) should be consulted for accuracy.
Metadata
Importance: 72/100wiki pagereference
Summary
Coherent Extrapolated Volition (CEV) is a theoretical AI alignment framework proposed by Eliezer Yudkowsky in 2004, describing how an artificial superintelligence should act based on what humans would want if they were more knowledgeable, rational, and morally mature. It aims to align AI with humanity's idealized preferences rather than current, potentially flawed desires.
Key Points
- •CEV proposes that AI should extrapolate an idealized human utility function reflecting what people would want under optimal epistemic and moral conditions.
- •Proposed by Eliezer Yudkowsky in 2004 as part of his Friendly AI research at MIRI (then SIAI).
- •The framework seeks to avoid alignment with transient, poorly informed, or individually conflicting human preferences.
- •Key challenge: aggregating diverse human preferences into a 'coherent' volition raises deep philosophical and technical difficulties.
- •Debated by figures including Nick Bostrom; critics question whether convergence of extrapolated preferences is feasible or well-defined.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Eliezer Yudkowsky | Person | 35.0 |
Cached Content Preview
HTTP 200Fetched Apr 9, 20266 KB
Coherent extrapolated volition - Wikipedia
Jump to content
From Wikipedia, the free encyclopedia
Ideal AI behavior if humans were maximally rational and knowledgeable
This article incorporates text from a large language model , which is prohibited in Wikipedia articles . It may include hallucinated information, copyright violations , claims not verified in cited sources, original research , or fictitious references . Any such material should be removed . ( October 2025 ) ( Learn how and when to remove this message )
Coherent extrapolated volition ( CEV ) is a theoretical framework in the field of AI alignment describing an approach by which an artificial superintelligence (ASI) would act on a benevolent supposition of what humans would want if they were more knowledgeable, more rational, had more time to think, and had matured together as a society, as opposed to humanity's current individual or collective preferences. [ 1 ] It was proposed by Eliezer Yudkowsky in 2004 as part of his work on friendly AI . [ 2 ]
Concept
[ edit ]
CEV proposes that an advanced AI system should derive its goals by extrapolating the idealized volition of humanity. This means aggregating and projecting human preferences into a coherent utility function that reflects what people would desire under ideal epistemic and moral conditions. The aim is to ensure that AI systems are aligned with humanity's true interests, rather than with transient or poorly informed preferences. [ 3 ]
In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.
— Eliezer Yudkowsky, Coherent Extrapolated Volition [ 2 ]
Debate
[ edit ]
Yudkowsky and Nick Bostrom note that CEV has several interesting properties. It is designed to be humane and self-correcting, by capturing the source of human values instead of trying to list them. It avoids the difficulty of laying down an explicit, fixed list of rules. It encapsulates moral growth, preventing flawed current moral beliefs from getting locked in. It limits the influence that a small group of programmers can have on what the ASI would value, thus also reducing the incentives to build ASI first. And it keeps humanity in charge of its destiny. [ 3 ] [ 2 ]
CEV also faces significant theoretical and practical challenges.
Bostrom notes that CEV has "a number of free parameters that could be specified in various ways, yielding different versions of the proposal." One such parameter is the extrapolation base (whose CEV is taken in
... (truncated, 6 KB total)Resource ID:
5d9024c39c4f4102 | Stable ID: sid_DaS0yej49y