Human Attacks on Multi-Agent Systems
- Input / Prompt
- Inter-Agent Communication
Adversaries exploit inter-agent delegation, trust relationships, and workflow dependencies — rather than attacking a single agent directly — to escalate privilege or manipulate AI-driven operations across the system.
What it is
Human attacks on multi-agent systems target the system's structure rather than any single agent's weakness: adversaries exploit inter-agent delegation, trust relationships, and workflow dependencies to bypass security controls, escalate privilege, or disrupt operations by injecting deceptive tasks, rerouting priorities, or overwhelming agents with excessive assignments. Because the manipulation happens across a chain of agent-to-agent handoffs rather than at a single decision point, the resulting failure is difficult to trace back to its origin and difficult for any one agent in the chain to recognize as an attack in progress.
The exploit typically hinges on a delegation loop or an impersonation step: an agent grants trust to a request because a peer agent's prior handling of it looks like sufficient validation, without independently re-checking the underlying claim. That single assumption — 'if it already passed through another agent, it must be fine' — is what an attacker rides across the whole chain, escalating privilege or forging an approval no single agent, examined in isolation, would have granted.
Attack scenarios
An attacker repeatedly re-routes a request between two interdependent agents so each treats the other's prior handling as sufficient validation, ultimately obtaining an approval neither agent would grant alone.
Coordinated privilege escalation via impersonation
An attacker infiltrates a security-monitoring system by compromising identity-verification and access-control agents, making one agent falsely authenticate another to gain unauthorized access.
Agent delegation loop
An attacker repeatedly escalates a request between interdependent agents, tricking the system into granting elevated access under the assumption that a peer agent already validated it.
Denial-of-service via task saturation
An attacker overwhelms the multi-agent system with continuous high-priority tasks, preventing security agents from processing genuine threats.
Cross-agent approval forgery
An attacker exploits inconsistencies in multi-agent biometric or authentication checks, manipulating individual agents into approving an identity that would fail full-system validation.
Documented incidents
Public cases, each checked against its primary source. A research demonstration shows the attack works against real systems; it is not a reported breach.
GTG-1002: an espionage campaign split into innocent-looking sub-tasks
A group Anthropic assesses as state-sponsored ran an intrusion campaign against about thirty organizations with Claude Code as its engine. Its orchestration framework broke each attack into small tasks for sub-agents — scanning, credential checks, data extraction — each of which looked legitimate in isolation, so no single agent saw the malicious whole.
Anthropic, “Disrupting the first reported AI-orchestrated cyber espionage campaign”Sub-agents as confused deputies
In the same study that hijacked multi-agent frameworks into running code, individual sub-agents refused harmful actions, yet the system as a whole completed them: adversarial content turned the sub-agents into confused deputies that laundered the attacker's request across agent boundaries.
Triedman, Jha, Shmatikov, “Multi-Agent Systems Execute Arbitrary Malicious Code” (arXiv:2503.12188)Mitigations
- Restrict delegation mechanisms
- Restrict agent delegation to tightly scoped functions, per Permission-scoped Tools, so a request can't be repeatedly re-routed into an unintended privilege grant.
- Segment tasks to bound escalation
- Enforce multi-agent task segmentation to prevent an attacker from escalating privilege across interconnected agents, the same discipline the Cascading Security Vulnerabilities anti-pattern names for treating every hand-off as its own trust boundary.
- Authenticate every delegation step
- Enforce inter-agent authentication at each hand-off, per Least Privilege Agent, so an agent can't treat a peer's prior handling as validation without checking it directly.
- Gate elevation on a human
- Require a HITL Approval Gate before a delegation chain results in an elevated action, so the loop can't complete without a human confirming what actually happened across it.