What Is Agent Tracing?

What is agent tracing?

Agent tracing records every step an AI agent takes during a run: reasoning turns, tool calls with their arguments and results, retrieved context, timing, tokens and cost, connected into a single replayable trace. It's what lets you identify which step caused a failure instead of only seeing the final bad output.

Tracing is the mechanism; agent observability is the discipline it serves. You trace in order to be able to observe.

Why do agents need tracing more than ordinary software does?

A conventional service fails loudly: an exception, a 500, a stack trace pointing at a line. An agent fails quietly and plausibly. It returns fluent prose built on a document it never retrieved, or it calls the right tool with a subtly wrong argument and carries on as if nothing happened. There is no exception to catch, because nothing technically errored.

Three properties make this worse.

Failures are mid-trajectory. The output is the last thing that happened, and by then the error is three steps upstream and laundered into confident language.

Runs are non-deterministic. 'Reproduce it' is not reliably available to you. The trace is often the only record that a particular failure ever occurred, which makes capture-by-default rather than capture-on-error the right setting.

The interesting state is in the arguments. Whether a tool call was correct usually depends on what was passed to it. A trace that records that a tool was called, without what it was called with, tells you very little.

What should an agent trace capture?

At minimum: the run's input, output, cost and outcome; each step in order with timing; every tool call including its arguments and result; retrieval queries and the chunks returned; and eval scores attached to individual steps. Tool arguments and retrieved context are the most commonly missed and cannot be added retroactively.

Each layer answers a different question. The run layer records input, final output, total tokens, cost, wall time and outcome, and answers whether the run succeeded and what it cost. The step layer records each reasoning turn and tool call, in order, with timing, and answers where the run went wrong. The tool-call layer records tool name, arguments, result, error and duration, and answers whether the call was correct or merely present. The retrieval layer records the query, the chunks returned and their scores, and answers whether the answer was grounded in what was actually retrieved. The scores layer attaches eval scores to the run and to individual steps, and answers which step caused the score to drop.

That last layer is the one teams skip, and it's the one that makes tracing pay for itself. A trace without scores is a debugging tool you use after someone complains. A trace with scores attached is a quality signal you can monitor, alert on, and gate releases with: the difference between forensics and detection.

What's the difference between tracing and monitoring for AI agents?

Tracing records connected steps within a single run and answers "what happened in this run?" Monitoring aggregates across many runs and answers "what's happening to quality overall?" Monitoring tells you quality dropped; tracing tells you why.

Logging is the third term, routinely conflated with both. Logging records discrete events: useful, unstructured, and disconnected, so you can't reconstruct a path from a log line. Tracing records connected steps within a single run, preserving order and causality. Monitoring aggregates across many runs.

You need all three. Tracing is the one most often missing, and the absence usually shows up as a team that can tell you their agent's quality dropped last Tuesday but not why. For the aggregate side, see What is Agent Monitoring?.

What's the difference between agent tracing and LLM tracing?

LLM tracing captures individual model calls: prompt, completion, tokens, latency. Agent tracing captures those plus the orchestration: which tool the agent chose, the arguments it passed, what came back, and how it reacted. For multi-step agents, most debugging information lives in the orchestration layer that LLM tracing doesn't cover.

The two are often used interchangeably; the scope differs. If your system is one prompt in and one completion out, LLM tracing is enough. Once the model is choosing between tools and chaining steps, most of what you need to debug lives in the orchestration layer rather than in any individual model call, which is exactly the part LLM tracing doesn't cover.

How do you get agent tracing right?

Instrument on a standard. OpenTelemetry-compatible tracing means your agent traces live alongside the rest of your system's telemetry rather than in a silo. OTel's GenAI semantic conventions now model an agent run as a span tree: an invoke_agent parent span with child chat spans for each model call and execute_tool spans for each tool invocation, which maps cleanly onto the layers above.

One caveat worth knowing before you standardise on them: as of late 2026 the gen_ai.* conventions are still marked 'Development', not Stable, and they were restructured in mid-2026. Instrument against them, since the direction of travel is clear and most vendors have moved, but expect attribute names to shift, and keep your instrumentation in a thin layer you can update in one place rather than scattered through your agent code.

Capture arguments and retrieved context, not just call names. This is the single most common gap, and it's not recoverable retrospectively.

Sample deliberately. Full-fidelity tracing on every production run gets expensive. Trace everything in development, sample in production, and always keep the full trace for any run that scored badly: those are the ones you'll want.

Attach scores to steps. See Agent Evals: A Practical Guide for what step-level scoring covers, and What Is an Eval Harness? for the scaffolding that produces those scores.

Prefactor traces agent runs at the step level with arguments and retrieved context captured by default, attaches eval scores to individual steps rather than only to the run, and links a failing score straight to the step that caused it.

Do I need agent tracing if I already have logging?

Yes. Logs record discrete, disconnected events: you can't reconstruct an agent's path from log lines because the ordering and causal links between steps aren't preserved. Tracing exists to preserve exactly that structure.

How much should I trace in production?

Trace everything in development. In production, sample, but always retain the full trace for any run that scored badly or was flagged by a user, since those are the runs you'll actually need. Non-determinism means an untraced failure may never reproduce.