Governing what acts on its own
An agent's behavior is composed at runtime, not fixed before deployment. That breaks four assumptions of ordinary software governance — and this page walks the first two: safety and security stop being separable, and assurance moves from a snapshot to a continuous loop.
Part XII of the reference names the threats and the controls that answer them. Governance answers the organizational question those raise: how do you assure a system whose behavior you cannot fully describe before it runs? The primary source is the OWASP Agentic Security Initiative's State of Agentic AI Security and Governance (v2.01, 2026).
Safety and security converge at the deployment layer
For most of software history these were separate disciplines with separate owners. Agentic autonomy collapses the distinction — but only at the deployment layer the deploying organization owns; model-level safety stays a distinct discipline owned by the provider.
Risks from trust-boundary violations: an attacker, malicious content, or unauthorized access enables influence that should not have been possible.
“Was a trust boundary crossed that should have held?”
Risks from normal operation: the system causes harm not because someone attacked it, but because its capabilities, defaults, or design permit harmful outcomes.
“Could this system cause harm through normal operation?”
| Trend | AI Safety | AI Security | Convergence effect |
|---|---|---|---|
| Expanding tool access | Larger blast radius when the agent misuses a capability on its own (including via indirect prompt injection from retrieved content) | Larger blast radius when an adversary triggers that misuse through injection | The same permission surface governs both failure modes |
| Reduced human oversight | Narrower window to catch a non-adversarial error before harm | Narrower window to detect adversarial manipulation before the agent acts | The same oversight gap enables both categories |
| Multi-agent architectures | A safety failure in one agent (hallucination, goal drift) propagates to others | A compromised agent becomes the attack vector against downstream agents | A single causal chain crosses the safety–security boundary |
| Agentic supply-chain growth | The agent invokes a poorly built tool that returns unreliable output | The agent invokes a malicious tool that exfiltrates data or poisons context | The same discovery and invocation path carries both risks |
The convergence scales with autonomy: for a low-autonomy or human-gated agent the categories stay usefully separable; for an agent with broad permissions and minimal oversight they are dimensions of a single risk surface — governed, monitored, and responded to together, because the telemetry that detects one produces the telemetry needed for the other.
Identity is the new control plane
A Non-Human Identity answers one question, once, at session start: is this entity allowed to connect? An agent reasons, delegates, and discovers tools at runtime, so the question becomes what is it doing right now, and is it still allowed to — governed at the moment of each action, not just at entry.
Provenance (the agent's code, weights, and runtime are intact), Attestation (the identity is what it claims — altered agents are denied tokens), and Intent (the action is within a declared, bounded purpose).
- Ephemeral, just-in-time credentials scoped to the current reasoning step and revoked on workflow completion — not long-lived static scopes.
- Delegation chains that cryptographically bind a request to the originating principal (OAuth Token Exchange, RFC 8693), so a downstream tool refuses to act as a confused deputy.
- A registry — an Agent Name Service (DNS-inspired, PKI-backed) — so decommissioned agents cannot linger as ghost agents with valid identities.
Each item is a pattern this reference already names, enforced at the credential layer: ephemeral scoping is Least Privilege Agent; attestation is the Audit Trail's provenance requirement.
Runtime governance, not static compliance
Pre-deployment certification loses meaning the moment an agent begins to run, accumulates context, loads tools dynamically, or modifies itself — the certified artifact is not the running system. Regulators agree: the EU AI Act (Art. 72) demands continuous lifetime monitoring, which functionally requires drift detection.
Four runtime capabilities most organizations still lack- Real-time behavioral monitoring, including plan-divergence detection — the agent's actual action sequence against its declared intent.
- Consequence-aware authorization — evaluating what the agent is doing, not merely inheriting the operator's standing permissions.
- Automated incident classification fast enough for compressed reporting windows (DORA 4h, NIS2 24h, RAISE 72h).
- Trajectory-level explainability — a runtime-composed trajectory was never anticipated at assessment time, so explainability that relies on static classification fails.
Governance becomes watching for when the agent departs from the bounded set of behaviors actually assessed and found compliant — and escalating when it does. The same monitoring infrastructure — tool-invocation logging, permission-chain auditing, anomaly detection, plan-divergence analysis — serves security detection and regulatory evidence at once.
Two heuristics for bounding agent risk
Both compress the guidance above into a design test you can apply before shipping. They share the same three properties.
Willison, 2025 — the three properties whose combination makes prompt injection exploitable end-to-end. Hold all three in one session and a single injection completes the whole chain. Break the triad by removing any one leg.
- 1
Access to private data
The agent can read sensitive or private information.
- 2
Exposure to untrusted content
Any text or data an attacker can get in front of the model.
- 3
Ability to communicate externally
A channel usable to exfiltrate or change state outside the boundary.
Meta, 2025 — the design constraint over the same three properties: within a session with no trusted human in the loop, satisfy at most two. An agent that needs all three must be gated by human approval. A pick-≤2 boundary, not a fix — it bounds exposure rather than eliminating it.
The frameworks
Where the durable frameworks attach
This project names half a dozen external frameworks in passing. They are not interchangeable: three govern a programme, three classify what an attacker did, and one is a scale. Only the taxonomies attach to a pattern — the management instruments attach to the organization around it.
| Framework | What question it answers | Where it attaches here |
|---|---|---|
| NIST AI RMF 1.0AI 100-1, 2023 · GenAI profile AI 600-1, 2024 | How do we organize risk work? Four functions — Govern, Map, Measure, Manage — plus a GenAI-specific profile of the same structure. | The maturity model |
| ISO/IEC 420012023 | Can an auditor certify how we run AI? The certifiable management-system standard. Not harmonized under the EU AI Act, so certification grants no presumption of conformity. | The governance programme around the system — not a pattern inside it |
| EU AI ActRegulation (EU) 2024/1689 | What does the law require? Obligations by risk class: human oversight (Art. 14), accuracy and robustness (Art. 15), serious-incident reporting (Art. 73). | The runtime-governance argument above |
| MITRE ATLAScontent v2026.06 | What did the attacker do to the AI system? An adversary-technique taxonomy for machine-learning and GenAI systems. | The threat catalogs |
| MITRE ATT&CK · CWEv19.1 · 4.20 | What did the attacker do to everything else, and which defect let them? The classic technique catalog and the root-cause weakness taxonomy. | Red teaming — once a tool call reaches a shell or a database, the incident is an ordinary one |
| CSA AI Controls Matrixv1.1, 2026 | Which controls do we claim, and who checks them? A control matrix in the lineage of the Cloud Controls Matrix. | The control crosswalk |
| OWASP AIVSSv0.8 | How bad is this one, in a number? Scoring for agentic risks, with an autonomy premium over the base score. | Prioritizing the threat catalogs |
No control mapping is maintained here. OWASP's Top 10 for LLM Applications 2026 carries one in its Appendix A — every entry mapped to ASI, DSGAI, ATLAS, ATT&CK, CWE, NIST and the CSA matrix, with each version pinned. It moves when the catalog moves; a second copy would only drift. The version stamps in the table come from that appendix and age the same way.
Security