Governing what acts on its own
An agent's behaviour is composed at runtime, not fixed before deployment. That breaks four assumptions of ordinary software governance — and this page walks the first two: safety and security stop being separable, and assurance moves from a snapshot to a continuous loop.
Part XII of the reference names the threats and the controls that answer them. Governance answers the organizational question those raise: how do you assure a system whose behaviour you cannot fully describe before it runs? The primary source is the OWASP Agentic Security Initiative's State of Agentic AI Security and Governance (v2.01, 2026).
Safety and security converge at the deployment layer
For most of software history these were separate disciplines with separate owners. Agentic autonomy collapses the distinction — but only at the deployment layer the deploying organization owns; model-level safety stays a distinct discipline owned by the provider.
Risks from trust-boundary violations: an attacker, malicious content, or unauthorized access enables influence that should not have been possible.
“Was a trust boundary crossed that should have held?”
Risks from normal operation: the system causes harm not because someone attacked it, but because its capabilities, defaults, or design permit harmful outcomes.
“Could this system cause harm through normal operation?”
| Trend | AI Safety | AI Security | Convergence effect |
|---|---|---|---|
| Expanding tool access | Larger blast radius when the agent misuses a capability on its own (including via indirect prompt injection from retrieved content) | Larger blast radius when an adversary triggers that misuse through injection | The same permission surface governs both failure modes |
| Reduced human oversight | Narrower window to catch a non-adversarial error before harm | Narrower window to detect adversarial manipulation before the agent acts | The same oversight gap enables both categories |
| Multi-agent architectures | A safety failure in one agent (hallucination, goal drift) propagates to others | A compromised agent becomes the attack vector against downstream agents | A single causal chain crosses the safety–security boundary |
| Agentic supply-chain growth | The agent invokes a poorly built tool that returns unreliable output | The agent invokes a malicious tool that exfiltrates data or poisons context | The same discovery and invocation path carries both risks |
The convergence scales with autonomy: for a low-autonomy or human-gated agent the categories stay usefully separable; for an agent with broad permissions and minimal oversight they are dimensions of a single risk surface — governed, monitored, and responded to together, because the telemetry that detects one produces the telemetry needed for the other.
Identity is the new control plane
A Non-Human Identity answers one question, once, at session start: is this entity allowed to connect? An agent reasons, delegates, and discovers tools at runtime, so the question becomes what is it doing right now, and is it still allowed to — governed at the moment of each action, not just at entry.
Provenance (the agent's code, weights, and runtime are intact), Attestation (the identity is what it claims — altered agents are denied tokens), and Intent (the action is within a declared, bounded purpose).
- Ephemeral, just-in-time credentials scoped to the current reasoning step and revoked on workflow completion — not long-lived static scopes.
- Delegation chains that cryptographically bind a request to the originating principal (OAuth Token Exchange, RFC 8693), so a downstream tool refuses to act as a confused deputy.
- A registry — an Agent Name Service (DNS-inspired, PKI-backed) — so decommissioned agents cannot linger as ghost agents with valid identities.
Each item is a pattern this reference already names, enforced at the credential layer: ephemeral scoping is Least Privilege Agent; attestation is the Audit Trail's provenance requirement.
Runtime governance, not static compliance
Pre-deployment certification loses meaning the moment an agent begins to run, accumulates context, loads tools dynamically, or modifies itself — the certified artifact is not the running system. Regulators agree: the EU AI Act (Art. 72) demands continuous lifetime monitoring, which functionally requires drift detection.
Four runtime capabilities most organizations still lack- Real-time behavioural monitoring, including plan-divergence detection — the agent's actual action sequence against its declared intent.
- Consequence-aware authorization — evaluating what the agent is doing, not merely inheriting the operator's standing permissions.
- Automated incident classification fast enough for compressed reporting windows (DORA 4h, NIS2 24h, RAISE 72h).
- Trajectory-level explainability — a runtime-composed trajectory was never anticipated at assessment time, so explainability that relies on static classification fails.
Governance becomes watching for when the agent departs from the bounded set of behaviours actually assessed and found compliant — and escalating when it does. The same monitoring infrastructure — tool-invocation logging, permission-chain auditing, anomaly detection, plan-divergence analysis — serves security detection and regulatory evidence at once.
Two heuristics for bounding agent risk
Both compress the guidance above into a design test you can apply before shipping. They share the same three properties.
Willison, 2025 — the three properties whose combination makes prompt injection exploitable end-to-end. Hold all three in one session and a single injection completes the whole chain. Break the triad by removing any one leg.
- 1
Access to private data
The agent can read sensitive or private information.
- 2
Exposure to untrusted content
Any text or data an attacker can get in front of the model.
- 3
Ability to communicate externally
A channel usable to exfiltrate or change state outside the boundary.
Meta, 2025 — the design constraint over the same three properties: within a session with no trusted human in the loop, satisfy at most two. An agent that needs all three must be gated by human approval. A pick-≤2 boundary, not a fix — it bounds exposure rather than eliminating it.
Security