When to reach for it
- Multiple components contribute to an answer.
- Error causes and latencies must be analyzed.
- Production operations require continuous monitoring.
Distributed Tracing.
Spans across the whole agent loop.
Agent runs, tool calls, and subprocesses are made visible as connected end-to-end traces. The cross-vendor standard is the OpenTelemetry GenAI semantic conventions (gen_ai.* span attributes), so a run instrumented once is portable across observability backends; a common sampling policy keeps all error traces and only a small fraction of successful ones.
In practiceA multi-step research agent emits OpenTelemetry spans for each tool call and sub-agent invocation, so an on-call engineer can pinpoint which retrieval step caused a P99 latency spike.
When it backfires
- The system remains minimal and local.
- Trace data creates unprotected privacy risks.
- Telemetry costs exceed the benefits.
The tradeoff
Deep diagnostic capability is achieved against telemetry and privacy management effort. Indispensable in production.
What it actually does.
Each step emits a span into one end-to-end trace.
Two ways this pattern will hurt you.
Token-flood traces
Every LLM call dumps its full prompt and response into the trace. Storage costs explode and the trace UI becomes unusable.
Fix · Sample full payloads at 1%. For the rest, log token counts, latency, and model name only. Summarize prompts if needed.
Trace IDs that don't survive tool boundaries
The agent passes a trace ID to a tool, but the tool calls another service that generates its own ID. The distributed trace breaks.
Fix · Propagate trace context via standard headers (e.g., W3C traceparent). Reject tools that don't forward the trace ID.
Where Distributed Tracing is native.
LangSmith / LangGraphrun trees + span exportNative
Microsoft Agent FrameworkOpenTelemetry exportNative
AWS X-Raydistributed-trace backendNative
Google Cloud Tracemanaged trace backendNative
Keep going