Sensitive Information Disclosure
- Output / Actuation
- Memory / State
The model inadvertently exposes confidential data — training data, retrieved documents, or another user's stored context — in its response.
What it is
Sensitive information disclosure covers any confidential content a model inadvertently reveals in its output — personal data, financial or health records, security credentials, proprietary business information, or fragments of its own training data and algorithms. A model that memorized part of its training set, or that holds an earlier user's input in context, can resurface it for an unrelated request; restricting what the model may repeat in the system prompt helps, but is not a hard boundary, since such restrictions can be bypassed by prompt injection. A multi-agent system widens the exposure with every surface it adds: memory pooled across sessions or tenants, a retrieval index fed by several agents, and hand-offs that pass raw context to the next agent all multiply the paths a secret can leak through. Treating each of those surfaces as needing its own access boundary — rather than trusting that data shared internally stays internal — is the core defense.
Kinds
- PII leakage
- Personal identifiable information surfaces in a response because the model was trained on it, retrieved it from shared context, or a user disclosed it earlier in the same conversation.
- Training data & algorithm exposure
- A poorly configured output reveals fragments of training data or proprietary model internals, enabling inversion or extraction attacks that reconstruct private inputs from the model's behavior.
- Business data disclosure
- A generated response inadvertently includes confidential business information the model had access to but the requester should not see.
Attack scenarios
A support agent with memory shared across tenants surfaces one customer's account details while answering a different customer's question.
Unintentional data exposure
Inadequate sanitization of shared context lets one user's response contain data belonging to a different user or session.
Targeted extraction via injection
An attacker crafts input designed to bypass an application's output filters and coax the model into repeating restricted information.
Leakage via training data
Sensitive content included in training or fine-tuning data without adequate review resurfaces later in unrelated outputs.
Cross-tenant memory bleed
A memory store shared by several agents has no per-tenant boundary, so one customer's stored account details surface while answering a different customer's question.
Mitigations
- Sanitize before it enters context
- Scrub or mask sensitive content before it enters training data, shared memory, or a retrieval index, and apply strict input validation so identifiable or confidential fields never enter the pipeline in the first place.
- Scope memory and data access
- Bind every agent's read access to only the memory partition and data sources its task needs, per Least Privilege Agent — an unpartitioned shared store is the most common cause of this class of leak.
- Filter every response
- Check outgoing content against an allow-list of permitted fields with Output Validation / Schema Enforcement before it reaches a user, rather than trusting the model's own restraint.
- Log every disclosure path
- Record what data crossed which boundary in the Audit Trail, so a leak is forensically traceable to its source rather than merely visible after the fact.