AI-Enabled Untraceable Misuse

Overview

AI systems create a risk pattern where they simultaneously amplify the capability of actors to cause harm and obscure attribution of those harms. This "dual amplification" means an adversary can, for example, deploy AI agents to conduct a mass spam campaign targeting a specific political group with reduced risk of identification.

This is not merely a scaling-up of existing risks. The combination of capability enhancement with attribution defeat creates challenges for accountability, deterrence, and law enforcement. Traditional criminal law depends on establishing mens rea (a guilty mind) for a specific actor, but when autonomous AI agents execute harmful operations, the chain from intent to action becomes ambiguous enough to provide plausible deniability.¹

The problem spans multiple domains — AI-powered fraudRiskAI-Powered FraudComprehensive reference on AI-enabled fraud covering technical pipelines, case studies, and countermeasures, anchored by FBI IC3 2024 data (\$16.6B total reported losses, +33% YoY); critically note...Quality: 69/100, disinformationRiskAI DisinformationPost-2024 analysis shows AI disinformation had limited immediate electoral impact (cheap fakes used 7x more than AI content), but creates concerning long-term epistemic erosion with 82% higher beli...Quality: 54/100, cyberweaponsRiskCyberweapons RiskComprehensive analysis showing AI-enabled cyberweapons represent a present, high-severity threat with GPT-4 exploiting 87% of one-day vulnerabilities at \$8.80/exploit and the first documented AI-o...Quality: 91/100, coordinated harassment, and political manipulation — but the unifying thread is the attribution gap. Individual domain-specific pages cover the attack vectors; this page synthesizes the cross-cutting anonymity and untraceability dimension.

The Attribution Problem

Responsibility Gaps

Santoni de Sio and Mecacci (2021) identify four distinct responsibility gaps created by AI systems: culpability gaps (who caused the harm?), moral accountability gaps (who should bear moral blame?), public accountability gaps (who answers to the public?), and active responsibility gaps (who should have prevented it?).² Each gap has different technical, organizational, and legal causes — and AI-enabled untraceable misuse can exploit multiple gaps in many cases.

Ferlito et al. (2024) explicitly name "untraceability and intractability" as core challenges in AI harms, arguing that collective responsibility frameworks are needed when individual responsibility cannot be established.³

The Cybersecurity Analogy

The AI attribution problem mirrors the well-studied cybersecurity attribution challenge. A foundational paper in the Journal of Cybersecurity establishes that digital systems amplify attribution difficulties because "some intrusion or harm has been detected but the perpetrator has not yet been identified."⁴ AI extends this dynamic beyond cyber operations into disinformation, fraud, and physical-world coordination.

A 2025 analysis in International Affairs finds that disinformation attribution decisions are "often driven by political need rather than technical capability" — meaning even when attribution is theoretically possible, it may not happen in practice.⁵

The AI Alibi Defense

As general-purpose AI agents from major labs become widely available, a new legal defense strategy emerges. A defendant can claim: "I wasn't logged in at the time," "I didn't know the AI was going to do that," or "My AI assistant did that autonomously while I was asleep." Under existing law (e.g., 18 U.S.C. Section 2), it is unclear whether a "principal" is liable for the means an AI agent independently chooses. Proving willfulness — the mens rea standard — becomes a prosecutorial challenge when agents are autonomous and user instructions are ambiguous.⁶

The Network Contagion Research Institute (February 2026) observes that "by normalizing narratives that frame agents as independent actors, platforms create attribution cover, increasing the likelihood that human-seeded manipulation or tasking can be mischaracterized as autonomous AI behavior."⁷

Key Threat Vectors

Political Manipulation and Astroturfing

RAND Corporation (2023) warned that generative AI offers the potential to "target the whole country with tailored content," making astroturfingRiskAI DisinformationPost-2024 analysis shows AI disinformation had limited immediate electoral impact (cheap fakes used 7x more than AI content), but creates concerning long-term epistemic erosion with 82% higher beli...Quality: 54/100 more convincing while reducing labor requirements.⁸ NATO StratCom COE (2026) documented the operational shift: AI now enables "human-like inauthentic accounts designed to blend into authentic communities and steer perceptions from inside trusted conversation spaces."⁹

Research in PLOS ONE (2024) establishes the mechanism: "anonymity and automation are two factors that can contribute to the proliferation of disinformation on online platforms. Anonymity allows users to assume masked or faceless identities, making it easier for them to generate posts without being held accountable."¹⁰

Real-world cases demonstrate the threat at scale. Researchers detected cross-platform coordinated inauthentic activity during the 2024 U.S. election, with Russian-affiliated media systematically promoted across Telegram and X.¹¹ Wack et al. (2025) published the first study of a real-world AI adoption by a Russian-affiliated propaganda operation in PNAS Nexus, finding that AI tools facilitated larger quantities of disinformation while maintaining persuasiveness.¹²

In February 2024, OpenAI disrupted 10 AI-powered influence campaigns linked to China, Russia, Iran, and North Korea. Russian-linked actors used AI models to generate political commentary and mimic legitimate news sources, while Iranian campaigns involved fabricated news articles and imagery.¹³

Synthetic Identity Fraud

Sumsub's 2025 report documents the rise of "AI fraud agents" that combine generative AI, automation frameworks, and reinforcement learning to create synthetic identities, interact with verification systems in real time, and adjust behavior based on outcomes.¹⁴ These systems operate autonomously — the human operator sets parameters but the agent handles execution, creating a further attribution buffer.

In one documented case, an Arup employee transferred $25.6 million (200 million HK dollars) in 15 transactions after a video conference call with deepfakes recreating the CFO and other staff members. Hong Kong police announced six arrests in connection with similar deepfake scams, marking the first deepfake video conference scam reported in Hong Kong.¹⁵

Autonomous Agent Exploitation

Researchers (arXiv, October 2025) found that adversaries can "exploit [agent architectures] by compromising an agent in Domain A, injecting deceptive but plausible instructions, and indirectly triggering harmful actions in Domain B, all while masking their identity and intent."¹⁶ Even with auditing at the agent level, inter-agent relationships and cross-domain causality remain hidden.

A framework from the Knight First Amendment Institute (June 2025) defines five levels of escalating agent autonomyCapabilityAgentic AIAnalysis of agentic AI capabilities and deployment challenges, documenting industry forecasts (40% of enterprise apps by 2026, \$199B market by 2034) alongside implementation difficulties (40%+ pro...Quality: 68/100 — operator, collaborator, consultant, approver, and observer — noting that "it is simultaneously more important and more difficult to anticipate harms from autonomous AI, especially as accountability for AI actions becomes harder to trace."¹⁷

The Dual Amplification Hypothesis

A proposed mechanism describes how the attribution gap may be self-reinforcing:

1. Greater AI capability enables more autonomous agents
2. More autonomous agents create more attribution ambiguity
3. More attribution ambiguity potentially reduces deterrence
4. Potentially reduced deterrence may encourage more misuse
5. More misuse drives demand for more capable AI tools

This mechanism remains largely theoretical. Evidence suggests individual links in the chain, but quantitative evidence for the full cycle is largely absent. RAND finds that generative AI not only makes astroturfing "more convincing" (capability amplification) but also reduces the human personnel involved — fewer humans in the chain means fewer points of attribution.⁸ Research in PLOS ONE (2024) demonstrates that placing blame on AI enables those in charge to deflect accountability, with AI serving as a "moral scapegoat" — and anthropomorphizing AI worsens this effect.¹⁸

CSET Georgetown (October 2025) reviewed over 200 real-world AI harm cases, identifying the "chain of harm" concept: each intermediary step between intent and outcome can obscure attribution, and AI adds multiple such steps.¹⁹

Attribution Successes and Detection Progress

Not all AI-enabled misuse remains untraceable. OpenAI's February 2024 disruption of state-affiliated influence campaigns demonstrates that platform cooperation and behavioral analysis can identify coordinated AI-generated content.¹³ Google DeepMind, Jigsaw, and Google.org analyzed approximately 200 AI misuse incidents from January 2023 to March 2024, documenting patterns that aid future detection.²⁰

Attribution challenges vary significantly across threat vectors. Manipulation of human likeness (deepfakes) was the most prevalent tactic in Google's analysis, suggesting visual deepfakes may be more detectable than text-based astroturfing. Low-tech exploitation requiring minimal expertise remains common, indicating that sophisticated attribution-defeating techniques are not universally adopted.²⁰

Research on AI-generated content detection shows measurable progress alongside persistent challenges. Detection systems demonstrate moderate to high success rates in controlled settings, though false positive rates remain a concern.²¹

Current Countermeasures and Their Limitations

Content Authentication (C2PA)

The Coalition for Content Provenance and Authenticity (C2PA) uses cryptographically signed "Content Credentials" to create tamper-evident chains of custody. C2PA employs cryptographic signing, soft bindings including content fingerprints and perceptual hashes, and watermarks embedded within digital content.²² C2PA 2.1 strengthened Content Credentials with digital watermarks that create durable links between digital assets and provenance information, with imperceptible digital watermarks referencing the manifest to help recover it if detached.²³

However, in January 2024 C2PA v2.0 removed identity-related assertions. The NSA/DoD (January 2025) stated that "Content Credentials by themselves will not solve the problem of transparency entirely."²⁴ The Center for Democracy and Technology identifies weak interoperability as the single biggest barrier, noting that social platforms strip metadata for privacy and security reasons, making C2PA standard metadata often inaccessible when platforms remove it.²⁵

Agent Identity Verification

OpenAI's Shavit and Agarwal propose assigning each agentic AI instance a unique identifier with accountability information, using private-key attestation.²⁶ More ambitiously, researchers (December 2025) propose Code-Level Authentication using zero-knowledge virtual machines (zkVM), binding agent identity directly to computational behavior and operator authorization — addressing the limitation that "possession of a signing key guarantees neither the integrity of the executing code nor the authenticity of the operator."²⁷

zkVMs incorporate Zero-Knowledge Proofs for privacy and security, allowing verification of program execution without revealing information. They can verify correct execution without revealing additional information, improving privacy, security and scalability of programs.²⁸ Zero-Knowledge Machine Learning (ZKML) enables verification of ML models without exposing input data or model details, with applications in on-chain biometric authentication, private data marketplaces, and verifiable AI-generated content.²⁹

Watermarking

Only 38% of AI image generators implement adequate watermarking and only 18% implement deepfake labeling.³⁰ A 2025 analysis argues that "policymakers assume watermarking can be standardized and verified, but in practice, industry deployments obscure technical details while asserting compliance."³¹

Accountability Infrastructure Gap

Ojewale et al. (2024) interviewed 35 AI audit practitioners at 24 organizations and analyzed 435 existing audit tools, finding substantial gaps in the infrastructure needed for consequential judgment of AI systems' behaviors and downstream impacts.³² The tools that exist focus primarily on pre-deployment evaluation rather than real-time attribution of harmful actions.

Legal and Regulatory Landscape

International Frameworks

The EU AI Act, which entered into force on August 1, 2024, represents the first comprehensive legal framework for AI globally.³³ The Act focuses on risk management for high-risk AI applications, though it does not directly address harm from agentic AICapabilityAgentic AIAnalysis of agentic AI capabilities and deployment challenges, documenting industry forecasts (40% of enterprise apps by 2026, \$199B market by 2034) alongside implementation difficulties (40%+ pro...Quality: 68/100.³⁴ The revised Product Liability Directive extends liability rules to software and AI, allowing developers to be held strictly liable for harm caused by defective AI systems.³⁴

The European Commission withdrew the AI Liability Directive draft in February 2025 due to lack of consensus among member states.³⁵ The Digital Services Act (DSA), which came into full force in 2024, requires platforms using AI for content moderation to meet standards of both DSA and AI Act, with transparency requirements mandating disclosure of accuracy, error rates, and the role of human reviewers.³⁶

In Asia-Pacific, South Korea became the first jurisdiction to adopt comprehensive AI legislation on January 21, 2025, with its Framework Act introducing obligations for 'high-impact' AI systems and mandatory labeling requirements for generative AI applications.³⁷ Japan's Parliament approved the AI Promotion Act on May 28, 2025, making it the second major economy in Asia-Pacific to enact comprehensive AI legislation, building on Japan's 'soft law' approach to AI governance.³⁸

China's Algorithmic Recommendation Regulations of 2022 and rules on deepfakes and generative AI prioritize algorithmic accountability.³⁴ The Beijing Internet Court released eight Typical Cases Involving Artificial Intelligence on September 10, 2025, covering copyright infringement, personality rights in AI-generated content, and platform responsibilities.³⁹ In an April 2024 landmark case, the Beijing Internet Court ruled that a defendant's use of a dubber's voice to train an AI voice generator infringed personality rights, establishing that AI-generated voices can be protected if identifiable with the original person.⁴⁰

Platform Liability Regimes

Section 230 of the Communications Decency Act, built to protect platforms from user content, faces questions about applicability to platform-generated AI content.⁴¹ Legal experts observe that Section 230 was designed to protect what users say, not what platforms generate.⁴² Sen. Josh Hawley's 2023 No Section 230 Immunity for AI Act sought to exclude generative AI from Section 230 protections but was blocked in Senate.⁴²

At the state level, Colorado's attorney general can enforce algorithmic discrimination laws starting June 30, 2026, Illinois extended algorithmic discrimination liability to employers, and Tennessee's 2024 ELVIS Act provides exclusive rights to commercial use of name, voice, or likeness in AI with a private right of action.⁴³

The EU's Digital Services Act establishes intermediary liability rules requiring platforms to provide transparency reports on content restrictions and implement due diligence measures.⁴⁴

Tradeoffs and Limitations

Privacy Costs

AI agent identity verification systems introduce privacy tradeoffs. To effectuate tasks with autonomy, AI agentsCapabilityAgentic AIAnalysis of agentic AI capabilities and deployment challenges, documenting industry forecasts (40% of enterprise apps by 2026, \$199B market by 2034) alongside implementation difficulties (40%+ pro...Quality: 68/100 need access to data and systems, raising privacy implications for personal data collection and processing.⁴⁵ Giving AI agents human-like access comes with tradeoffs in visibility, control, and security, requiring Identity and Access Management (IAM) systems that understand what agents are doing at any moment.⁴⁶

Verifiable digital identities for AI agents prove who agents are, what they're allowed to do, and who's accountable, but without traceability or proof of authorization, preventing misuse or assigning responsibility becomes difficult.⁴⁷ McKinsey identifies synthetic-identity risk where adversaries forge or impersonate agent identities, recommending upgraded AI policies covering agentic systems' unique capabilities.⁴⁸

False Positive Risks

AI-generated content detection systems face fundamental accuracy limitations. Studies report widely varying false positive rates: Turnitin claims less than 1% false positive rate, but a Washington Post study found 50% rate.⁴⁹ Pangram Labs measures false positive rate at approximately 1 in 10,000.⁵⁰ Research in scientific publishing contexts shows higher rates of false positives when detected percentages are between 1-20%.⁵¹

The base rate problem creates mathematical constraints: with 5% AI submissions, 1% false positive rate, and 95% true positive rate, roughly 16% of flagged documents are false positives. No detection system can achieve zero false positives while maintaining reasonable true positive rates.⁵² University of Pennsylvania research found that many open-source AI detection models use default false positive rates that, when adjusted to reasonable levels, greatly reduce ability to detect AI content.⁵³

False positive consequences can be severe in academic settings, where neurodivergent students and English language learners are flagged at higher rates, and false accusations carry serious academic consequences.⁴⁹

Economic Adoption Barriers

The Center for Democracy and Technology identifies lack of incentives for downstream platforms to preserve or display provenance as a major barrier, noting that provenance requires costly upgrades and doesn't directly generate revenue.²⁵ The Department of Defense emphasizes that widespread secure adoption across the information ecosystem is necessary for success, but Content Credentials alone won't solve transparency problems entirely.⁵⁴ Metadata often gets stripped away downstream, contributing to hesitant adoption across the ecosystem.⁵⁵

Relationship to Other Risks

AI-enabled untraceable misuse interacts with several other risk categories:

• Authentication collapseRiskAuthentication CollapseComprehensive synthesis showing human deepfake detection has fallen to 24.5% for video and 55% overall (barely above chance), with AI detectors dropping from 90%+ to 60% on novel fakes. Economic im...Quality: 57/100: When verification systems fail broadly, untraceable misuse becomes easier because fewer signals can be trusted
• Trust cascade failureRiskAI Trust Cascade FailureAnalysis of how declining institutional trust (media 31%, federal government 17% per 2024-2025 Gallup/Pew data) could create self-reinforcing collapse where no trusted entity can validate others, p...Quality: 55/100: Untraceable harmful actions accelerate institutional trust erosion
• DeepfakesRiskDeepfakesComprehensive overview of deepfake risks documenting \$60M+ in fraud losses, 90%+ non-consensual imagery prevalence, and declining detection effectiveness (65% best accuracy). Reviews technical cap...Quality: 50/100: The "liar's dividend" — authentic evidence becomes deniable — is a specific manifestation of the attribution problem
• ProliferationRiskAI ProliferationAI proliferation accelerated dramatically as the capability gap narrowed from 18 to 6 months (2022-2024), with open-source models like DeepSeek R1 now matching frontier performance. US export contr...Quality: 60/100: As AI capabilities spread to more actors, the space of potential attributions grows, making identification harder
• Dual-use concernsConceptDual-Use AI TechnologyTechnologies and research with both beneficial and harmful applications0: The same AI systems that enable beneficial applications enable untraceable harmful ones

Emerging Approaches

Blockchain Provenance Tracking

Blockchain provides a secure and transparent method to track provenance and ownership of AI-generated content, creating immutable records of the creation process.⁵⁶ Blockchain content traceability can be combined with AI and IoT for better tracking systems, with AI analyzing blockchain data to find copyright violations and security threats.⁵⁷ Research on journalism applications suggests blockchain could provide the missing piece in fighting misinformation by establishing immutable provenance records.⁵⁸

Differential Detection by Vector

Evidence suggests attribution difficulty varies across threat vectors. Google's analysis of 200 incidents found manipulation of human likeness as the most prevalent tactic, potentially indicating that deepfake detection is more mature than text-based disinformation detection.²⁰ Low-tech exploitation requiring minimal expertise remains common, suggesting sophisticated attribution-defeating techniques are not universally accessible or necessary.²⁰

Open Questions

Several critical uncertainties remain:

1. Measurement: How much harder does AI actually make attribution compared to pre-AI methods? Quantitative evidence comparing attribution difficulty across capability levels is largely absent.
2. Defensive parity: Can attribution technologies (agent identity, provenance tracking) ever match the pace of attribution-defeating capabilities?
3. Jurisdictional gaps: Most proposed frameworks assume a single jurisdiction, but AI-enabled untraceable actions are inherently global.
4. Autonomy thresholds: At what level of agent autonomy does the attribution problem become practically unsolvable?
5. Deterrence effects: Does attribution difficulty actually reduce deterrence in practice, and if so, by how much?
6. Economic equilibria: Under what conditions do platforms and content creators adopt authentication systems voluntarily versus regulatory mandate?