Improper Output Handling
- Output / Actuation
Model-generated content is passed downstream — into a shell, a database query, a browser, another agent — without adequate validation or sanitisation, so the model's text becomes an execution path.
What it is
Improper output handling is a gap between what a model generates and what a downstream system does with it: insufficient validation or sanitisation of the output before it reaches a shell, a database, a browser, or another agent turns generated text into an execution path. Because the output is ultimately shaped by the prompt, an attacker who controls the input indirectly controls what gets executed downstream — the model is an untrusted intermediary, not a client whose text can be trusted by default. The risk compounds in an agentic system, where one agent's output routinely becomes another agent's input: a downstream agent that trusts a peer's generated text the way it trusts its own reasoning inherits every unvalidated payload the upstream agent produced, with no record of where it originated.
Kinds
- Code / command injection
- Generated text is passed directly into a system shell or a function like exec or eval, letting a crafted prompt achieve remote code execution through the model as the delivery mechanism.
- Markup / script injection
- Generated JavaScript or Markdown is rendered by a browser without sanitisation, producing cross-site scripting from content the model itself synthesized.
- Structured-query injection
- A generated SQL fragment or similar structured query is concatenated into a live statement without parameterisation, letting an injected character escape the intended query scope.
Attack scenarios
An agent's generated SQL fragment is concatenated directly into a live query, and an injected character lets the query read outside the caller's own records.
Unvalidated inter-component response
A general-purpose model passes its response straight to a privileged extension without output validation, and the extension acts on content it should have treated as untrusted.
Prompt-injection-driven exfiltration
A webpage-summarizer agent processes a hidden instruction embedded in the page, encodes sensitive conversation content, and sends it to an attacker-controlled server, with no output validation catching the outbound payload.
Unscrutinized generated SQL
A chat feature lets a user request a database query in natural language; the model's generated SQL is executed without review, and a destructive request deletes production tables.
Reflected script in generated content
A crafted prompt causes the model to return an unsanitized script payload that executes in a victim's browser once rendered, because the application trusted the model's output as safe HTML.
Mitigations
- Treat model output as untrusted input
- Apply zero-trust validation to every model response before it reaches a backend function — the same scrutiny given to any other untrusted client, matching Output Validation / Schema Enforcement.
- Encode output for its destination
- Apply context-aware encoding — HTML encoding for web content, parameterized queries for SQL, path sanitisation for file operations — based on where the output is actually used, never a single generic filter.
- Sandbox anything the output can execute
- Run generated code or commands inside Sandbox Execution, isolated from the host, so a successful injection has nothing privileged to reach.
- Log and monitor outbound patterns
- Watch for unusual patterns in what the model emits — the Audit Trail's record of what left the system turns a single exploit attempt into a detectable pattern.