Back
DARPA Assured Autonomy Program
webdarpa.mil·darpa.mil/program/assured-autonomy
DARPA's Assured Autonomy program is a U.S. government research initiative relevant to technical AI safety, particularly formal verification of learning systems in high-stakes autonomous contexts; notable as a real-world institutional effort to operationalize safety guarantees.
Metadata
Importance: 45/100homepage
Summary
DARPA's Assured Autonomy program aims to develop methods for continuous assurance of learning-enabled autonomous systems operating in dynamic environments. It focuses on providing mathematical guarantees and formal verification for machine learning components in safety-critical autonomous systems such as aircraft and ground vehicles. The program seeks to ensure that autonomous systems behave safely and as intended even as they adapt and learn.
Key Points
- •Develops techniques to provide formal, provable guarantees on the safety and reliability of learning-enabled autonomous systems
- •Targets safety-critical military applications including autonomous aircraft and ground vehicles operating in complex environments
- •Addresses the challenge of verifying behavior of machine learning components that adapt over time
- •Aims to integrate automated verification and testing throughout the system development and operation lifecycle
- •Represents government-funded research bridging AI safety, formal methods, and real-world autonomous systems deployment
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Autonomous Weapons Escalation Model | Analysis | 62.0 |
Cached Content Preview
HTTP 200Fetched Apr 9, 20265 KB
Assured Autonomy | DARPA
Skip to main content
Official websites use .mil
A
.mil website belongs to an official U.S.
Department of War organization.
Secure .mil websites use HTTPS
A
lock (
Lock
Locked padlock icon
) or https:// means you’ve safely connected to the .mil website. Share sensitive information only on official, secure websites.
Breadcrumb
Home
Research
Programs
Assured Autonomy
Assured Autonomy
Summary
Autonomy refers to a system’s ability to accomplish goals independently, or with minimal supervision from human operators in environments that are complex and unpredictable.
Autonomous systems are increasingly critical to several current and future Department of Defense (DoD) mission needs. For example, the U.S. Army Robotics and Autonomous Systems (RAS) strategy report for 2015-2040 identifies a range of capability objectives, including enhanced situational awareness, cognitive workload reduction, force protection, cyber defense, logistics, etc, that rely on autonomous systems and higher levels of autonomy.
Tremendous advances have been made in the last decade in constructing autonomous Cyber Physical Systems (CPS), as evidenced by the proliferation of a variety of unmanned systems: air, ground, sea, and undersea vehicles. These advances have been driven by innovations in several areas, such as sensor and actuator technologies, computing technologies, control theory, design methods and tools, modeling and simulation technologies, among others. In spite of these advances, deployment and broader adoption of such systems in safety-critical DoD applications remains challenging and controversial.
Several factors impede the deployment and adoption of autonomous systems:
In the absence of an adequately high level of autonomy that can be relied upon, substantial operator involvement is required, which not only severely limits operational gains, but creates significant new challenges in the areas of human-machine interaction and mixed initiative control.
Achieving higher levels of autonomy in uncertain, unstructured, and dynamic environments, on the other hand, increasingly involves data-driven machine learning techniques with many open systems science and systems engineering challenges.
Machine learning techniques widely used today are inherently unpredictable and lack the necessary mathematical framework to provide guarantees on correctness, while DoD applications that depend on safe and correct operation for mission success require predictable behavior and strong assurance.
Historically, assurance has been approached through design processes following rigorous safety standards in development, and demonstrating compliance through
... (truncated, 5 KB total)Resource ID:
822fa24c1716fe53 | Stable ID: sid_V6P2uugAKM