Back
AI Red Teaming: Applying Software TEVV for AI Evaluations
governmentCredibility Rating
4/5
High(4)High quality. Established institution or organization with editorial oversight and accountability.
Rating inherited from publication venue: CISA
Official CISA guidance applying traditional software testing frameworks to AI red teaming; relevant for practitioners in government, critical infrastructure, and organizations seeking authoritative federal guidance on AI security evaluation practices.
Metadata
Importance: 52/100guidance documentreference
Summary
A CISA publication exploring how traditional software Testing, Evaluation, Verification, and Validation (TEVV) methodologies can be adapted and applied to AI red teaming exercises, helping organizations systematically evaluate AI system safety and security. The resource bridges established cybersecurity practices with emerging AI evaluation needs for government and critical infrastructure contexts.
Key Points
- •Adapts established software TEVV frameworks to address the unique challenges of evaluating AI systems for safety and security vulnerabilities.
- •Red teaming for AI requires expanded scope beyond traditional software testing, including adversarial prompting, model behavior analysis, and failure mode identification.
- •CISA positions AI red teaming as essential for critical infrastructure operators deploying AI in high-stakes environments.
- •Provides practical guidance for organizations seeking to integrate AI evaluations into existing cybersecurity assessment programs.
- •Emphasizes collaboration between AI developers, security professionals, and government stakeholders in evaluation processes.
Cited by 3 pages
| Page | Type | Quality |
|---|---|---|
| AI Evaluations | Research Area | 72.0 |
| Third-Party Model Auditing | Approach | 64.0 |
| Red Teaming | Research Area | 65.0 |
Cached Content Preview
HTTP 200Fetched Apr 9, 202614 KB
AI Red Teaming: Applying Software TEVV for AI Evaluations | CISA
Skip to main content
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock ( Lock A locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Due to the lapse in federal funding, this website will not be actively managed. Read More
no-cost Cyber Services Secure by design Secure Your Business Shields Up Report A Cyber Issue
Share:
Blog
AI Red Teaming: Applying Software TEVV for AI Evaluations
Released November 26, 2024
By: Jonathan Spring, Deputy Chief AI Officer and Divjot Singh Bawa, Strategic Advisor
Related topics:
Cybersecurity Best Practices , Cyber Threats and Response
As the National Coordinator for critical infrastructure security and resilience, CISA is responsible for facilitating a Secure by Design approach to AI-based software across the digital ecosystem and helping protect critical infrastructure from malicious uses of AI. To effectively mitigate against critical failures, physical attacks, and cyberattacks, AI software developers must prioritize conducting rigorous safety and security testing to understand how an AI system can fail or be exploited.
AI red teaming is a foundational component of the safety and security evaluations process.
This blogpost demonstrates that AI red teaming must fit into the existing framework for AI Testing, Evaluation, Validation and Verification (TEVV). Additionally, the post explains how and why AI TEVV must fit into software TEVV, ensuring AI systems are fit for purpose . While there are differences in the specific software tools used, AI TEVV—despite common misconceptions—must be treated under software TEVV from a strategic and operational perspective.
This assertion is grounded in the fact that TEVV has been used for more than four decades to improve the safety and security of software. 1 Experts working on AI evaluations should avoid reinventing the wheel and build upon lessons the software security community has learned through developing and improving guidance and requirements.
Framing AI Red Teaming in the Context of TEVV
AI red teaming is the third-party safety and security evaluation of AI systems; AI red teaming is a subset of AI Testing, Evaluation, Verification and Validation (TEVV).
AI TEVV , a broader risk-based approach for the external testing of AI systems, has been developed and operationalized by o
... (truncated, 14 KB total)Resource ID:
6f1d4fd3b52c7cb7 | Stable ID: sid_NEMTJ2Ilvp