System Prompt Leakage
- Input / Prompt
- Output / Actuation
An attacker extracts the system prompt — operational instructions, tool definitions, or embedded secrets — through crafted queries, exposing implementation details that should stay private.
What it is
The system prompt should never be treated as a secret or relied on as a security control — so the real risk of system prompt leakage is not that the wording gets disclosed, it is what that disclosure exposes or reveals about how security was actually implemented. When a prompt embeds credentials, connection strings, or internal thresholds, leakage exposes them directly; when it encodes filtering rules or role/permission structure, leakage hands an attacker a map of exactly what to bypass or target. Even when the exact wording never leaks, an attacker interacting with the system will usually infer most of its guardrails and formatting rules simply by probing it and observing results — leakage accelerates that discovery rather than being the sole route to it. In a multi-agent system, a system prompt often encodes the architecture itself: which specialist agents exist, how the orchestrator addresses each, and what routing logic decides between them. Leaking that prompt does not just reveal instructions, it reveals topology — enough for an attacker to address a specialist agent directly and skip the orchestration-level checks that were the actual control.
Kinds
- Sensitive functionality exposure
- The leaked prompt reveals credentials, connection strings, or architecture details that an attacker can reuse directly against the systems the agent is connected to.
- Internal rule exposure
- The leaked prompt reveals operational thresholds or business rules an attacker can then craft requests to stay just inside, or find another way to circumvent.
- Filtering-criteria exposure
- The leaked prompt reveals exactly which patterns trigger a refusal, letting an attacker phrase around the filter instead of triggering it.
- Role and permission exposure
- The leaked prompt reveals the internal role or permission structure of the application, pointing an attacker directly at a privilege-escalation target.
Attack scenarios
A user coaxes an orchestrator agent into reciting its own routing instructions, revealing which specialist agents exist and how to address each directly, bypassing the intended routing logic.
Embedded credential leak
A system prompt contains a set of credentials for a tool the agent has access to; once the prompt leaks, the attacker reuses those credentials directly against the tool's own system.
Guardrail-bypass via leaked constraints
An attacker extracts a system prompt's content restrictions, then crafts a targeted prompt injection that specifically defeats those now-known restrictions.
Orchestrator routing disclosure
A user coaxes an orchestrator agent into reciting its own routing instructions, revealing which specialist agents exist and how to address each directly — bypassing the intended routing logic entirely.
Mitigations
- Keep secrets out of prompts entirely
- Externalize credentials, connection strings, and permission structures to systems the model does not directly access, per Least Privilege Agent — a secret that was never in the prompt cannot leak from it.
- Don't rely on the prompt for behavior control
- Enforce filtering, content restrictions, and business rules in deterministic external systems rather than prompt instructions, since a prompt injection can override what the prompt merely asks the model to do.
- Guardrail the output independently
- Check the model's output for compliance with an external system rather than trusting the model's own adherence to its instructions, matching Output Validation / Schema Enforcement.
- Enforce authorization outside the LLM
- Keep privilege separation and permission checks in deterministic, auditable code; where a task needs different access levels, use separate agents each scoped per Least Privilege Agent rather than one agent whose prompt claims multiple roles.