When to reach for it
- Agents act autonomously in socially or commercially sensitive environments.
- Transparency regarding decision-making is required.
- A fleet of agents needs a supervisory layer that does not require a human at every step.
Controller.
Polices the agent's actions against policy as the run unfolds.
Continuously monitors the behaviour of the agent and transparently aligns it with ethical principles and compliance rules. Realised at scale as a governance agent (watching for policy violations) or security agent (flagging anomalous behaviour) supervising other agents, escalating to a human only on a trip.
In practiceA financial-advice agent passes every draft recommendation through a compliance controller that checks it against a regulatory ruleset, halting the run and alerting a reviewer on any rule trip.
Without itWithout a controller, policy violations from autonomous agents in sensitive domains surface only after consequences — a fine, a harmful disclosure, or a regulatory breach — rather than at the point of the offending action.
When it backfires
- The agent operates exclusively in safe, low-risk sandbox environments with no external impact.
The tradeoff
Stricter compliance is gained at the expense of operational overhead and complex rule definitions.
What it actually does.
A controller polices actions against policy.
Two ways this pattern will hurt you.
The controller silently blocks; the agent retries forever
The controller rejects a call but returns a generic error. The agent treats it as a transient failure and retries with the same payload.
Fix · Classify rejections as policy-final or transient. Policy-final must surface as a typed signal the agent can react to — not as a retryable error.
An action path that skips the controller
The controller wraps the main tool calls, but a side path — a directly-invoked tool, a sub-agent, a cached action — reaches the outside world without passing through it. The one unguarded path is where the incident happens.
Fix · Enforce the controller at a single choke point every side-effecting action must traverse, rather than decorating call sites one by one; fail closed if an action arrives unrouted.
Where Controller is native.
AllcustomAdaptable
Keep going