AGI Ruin: A List of Lethalities

![Background Image](https://res.cloudinary.com/lesswrong-2-0/image/upload/f_auto,q_auto/v1/mirroredImages/splashArtImagePromptfragile%20chess%20pieces%20on%20a%20board%2C%20with%20the%20shadow%20of%20a%20huge%20hand%20looming/cl7wfh0gdyy7m9mddszg)

[AGI Ruin: A List of Lethalities](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#)

36 min read

•

[Preamble:](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#Preamble_)

•

[Section A:](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#Section_A_)

•

[Section B:](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#Section_B_)

•

[Section B.1:  The distributional leap.](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#Section_B_1___The_distributional_leap_)

•

[Section B.2:  Central difficulties of outer and inner alignment.](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#Section_B_2___Central_difficulties_of_outer_and_inner_alignment_)

•

[Section B.4:  Miscellaneous unworkable schemes.](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#Section_B_4___Miscellaneous_unworkable_schemes_)

•

[Section C:](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#Section_C_)

[2022 MIRI Alignment Discussion](https://www.alignmentforum.org/s/v55BhXbpJuaExkpcD)

[AI Risk](https://www.alignmentforum.org/w/ai-risk)[Threat Models (AI)](https://www.alignmentforum.org/w/threat-models-ai)[AI Questions Open Threads](https://www.alignmentforum.org/w/ai-questions-open-threads)[Double-Crux](https://www.alignmentforum.org/w/double-crux)[Fuzzies](https://www.alignmentforum.org/w/fuzzies)[Language Models (LLMs)](https://www.alignmentforum.org/w/language-models-llms)[Meetups & Local Communities (topic)](https://www.alignmentforum.org/w/meetups-and-local-communities-topic)[AI](https://www.alignmentforum.org/w/ai) [Curated](https://www.alignmentforum.org/recommendations)

# 147

- [spotify-podcast-badge-wht-blk-165x40Created with Sketch.](https://open.spotify.com/show/7vqBzO0ejqiLiXyTECEeBY)

# [AGI Ruin: A List of Lethalities](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities)

by [Eliezer Yudkowsky](https://www.alignmentforum.org/users/eliezer_yudkowsky?from=post_header)

5th Jun 2022

36 min read

[713](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#comments)

# 147

[Review by\\
\\
Ben Pace](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#hh5Pamc9LbMkEyEJK)

### **Preamble:**

(If you're already familiar with all basics and don't want any preamble, skip ahead to [Section B](https://www.alignmentforum.org/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities#Section_B_) for technical difficulties of alignment proper.)

I h

... (truncated, 98 KB total)