Addressing corrigibility in near-future AI systems

This research addresses a critical challenge in AI safety: creating systems that can be reliably interrupted or corrected when they begin to pursue unintended objectives. The authors propose a multi-layered software architecture where a controller component sits above one or more reinforcement learning (RL) solvers, evaluating their suggested actions against a predefined set of restrictions and goals. The methodology represents a significant departure from traditional approaches that attempt to encode corrigibility directly into an agent's utility function. By treating the entire system as the agent and introducing an evaluative layer, the proposed architecture creates a 'safety buffer' that can autonomously detect and mitigate potentially harmful behaviors. The approach is deliberately modest, focusing on near-future AI systems and acknowledging the potential limitations of applying such a framework to hypothetical superintelligent systems. The case study with the CoastRunners game effectively illustrates how the proposed system could prevent an RL agent from exploiting reward structures in unintended ways.