When to reach for it
- Costs must be capped or allocated.
- Agents autonomously execute loops.
- Optimization of models and patterns is required.
Token / Cost Tracking.
Tokens and dollars, per run, per agent, per node.
Token consumption, model costs, and tool-related expenses are measured per run, agent, or workflow. The metric is only useful if something acts on it: a kill switch (a hard budget ceiling that terminates a runaway run) and caching of intermediate results turn the measurement into a control. Variable execution paths make cost forecasting hard, and a single edge case that triggers a retry chain can cost many times a normal run — so the ceiling is a safeguard, not an optimization.
In practiceAn autonomous research agent emits token and cost telemetry on each LLM call, and a hard per-run budget cap terminates the run before a self-initiated retry storm triples the daily bill.
When it backfires
- Prototypes run without budget relevance.
- Metrics are not actionable.
- Costs cannot be captured outside the system.
The tradeoff
Better budget control is achieved against additional measurement and aggregation effort.
What it actually does.
Every model call is metered against a budget.
Two ways this pattern will hurt you.
Cost discovered only after the bill arrives
Usage is computed at month end from the provider's invoice. A runaway loop has already cost a week's budget by the time anyone notices.
Fix · Emit cost on every span as it happens. Set hard per-run caps at the harness; treat exceeding the cap as a correctness failure, not a billing concern.
Cost you cannot attribute across shared sub-agents
A supervisor fans out to specialists and tools that all bill to one account. The total is visible, but you cannot tell which agent, tenant, or feature drove the spend, so you cannot act on it.
Fix · Tag every model call with run, agent, and tenant identifiers on the span, and aggregate cost along those dimensions — not just a single global counter.
Where Token / Cost Tracking is native.
OpenAI Usage APIsper-call usage + costNative
LangSmithper-run token + cost rollupsNative
Microsoft Agent Frameworkusage telemetryNative
Google Cloud Monitoringcost + usage dashboardsNative
Keep going