Human Compatible: Artificial Intelligence and the Problem of Control

Summary

landmark book argues that the standard model of AI—machines optimizing fixed objectives—is fundamentally flawed and proposes a new framework based on machines that are uncertain about human preferences and defer to humans. It presents the case that beneficial AI requires solving the value alignment problem and outlines a research agenda centered on cooperative inverse reinforcement learning and provably beneficial AI.

Key Points

• •Critiques the standard AI model of fixed-objective optimization as inherently dangerous, as machines will pursue objectives regardless of unintended consequences.
• •Proposes three core principles for beneficial AI: machines must be uncertain about human preferences, preferences can be inferred from human behavior, and machines should defer to humans.
• •Introduces the concept of 'human compatible' AI as a new paradigm replacing the Turing test goal of human-like intelligence with the goal of beneficial intelligence.
• •Argues that the AI control problem is the defining challenge of our time and that solving it requires fundamental changes to how AI systems are designed.
• •Accessible to general audiences while grounded in technical AI research, written by a leading AI textbook author and researcher.