The math that breaks production agents
If each step of a ten-step agent workflow succeeds 85% of the time, the probability that all ten steps succeed is 0.85 to the power of ten: roughly 20%. That is not a hypothetical. It is the compounding reality that separates benchmark scores from production reliability, and it explains why a March 2026 survey of 650 enterprise technology leaders found that 78% of enterprises have AI agent pilots but fewer than 15% have reached production scale.
The standard response is to improve the model or prompt, then re-run the benchmark. Neither fix addresses the structural problem: evaluating final outputs misses every intermediate decision that led there. An agent can produce a correct-looking answer through a sequence of flawed reasoning steps, and a correct process can still fail at the last step. Neither case is visible when you only inspect the end state.
Trajectory-level evaluation measures the full reasoning path, step by step, and scores quality across the whole sequence rather than at the endpoint alone. It is the difference between auditing a financial ledger entry and auditing every transaction that produced it.
Why step-level accuracy compounds silently
Most evaluation setups treat each agent action in isolation: did the tool call return valid output, did the LLM respond coherently, did the final answer match the reference? These checks catch obvious failures. They do not catch the class of failures that dominate production: a correct step built on a misclassified earlier step, a valid tool call made in the wrong sequence, a plausible intermediate conclusion that redirects the entire workflow toward the wrong goal.
This is what the research community calls cascading failure in agent step sequences. An early error does not announce itself. It propagates through subsequent steps, each of which may individually pass quality checks, until the workflow produces an outcome that is coherent but wrong.
A 2025 Scale AI report found that 34% of enterprises experienced unexpected agent behavior changes following a model update, with 12% experiencing production incidents severe enough to require human intervention. The incidents were not caused by sudden model collapse. They were caused by small shifts in intermediate reasoning that accumulated across multi-step workflows before anyone noticed.
How production teams instrument trajectories
The companies that have moved agents into production at scale have converged on a common pattern: record every span of the agent's execution, score intermediate steps against expected behavior schemas, and build gates that halt or escalate before errors compound.
Replit integrated trace-based observability into their AI Agent product during the alpha phase, specifically to search long-running traces and pinpoint where users got stuck in multi-turn code generation workflows. Rather than asking "did the agent produce good code," they asked "at which step did the reasoning diverge." That reframing let them identify bottlenecks in agent trajectories and correct them iteratively rather than reactively.
Uber runs approximately 60,000 agent task executions per week through their MCP Gateway. To manage that volume, they built a dedicated Agent Studio that visualizes multi-agent graphs, execution traces, and step-level failures. The result by early 2026: 84% of Uber developers using agentic coding tools daily, with checkpoint-based error recovery catching trajectory deviations before they reach downstream systems. That kind of scale is not achievable by reviewing final outputs after the fact.
LinkedIn structured their candidate sourcing and matching system as a hierarchical multi-agent graph, deliberately breaking workflows into the smallest possible tasks to limit how far a per-step error could propagate before it was caught. The architectural choice was an evaluation strategy: smaller tasks produce shorter trajectories, shorter trajectories have fewer compounding steps.
Anthropic's operational data shows that human override rate is a leading indicator of agent degradation: when override rates climbed from 5% to 12% over two weeks, system-level quality incidents followed within one week. Override rate is a trajectory-level signal. A human choosing to intervene mid-workflow is responding to a perceived error in the agent's reasoning path, not just the final output. Tracking when and where overrides occur maps directly onto the steps where trajectory quality is lowest.
What trajectory evaluation actually requires
Instrumenting trajectories in production requires three things: span recording at the step level, scoring logic that runs against intermediate states, and a schema that defines expected behavior at each point in the workflow.
Span recording means capturing every tool call, every LLM completion, every routing decision, and every state transition, not as a log but as a structured trace with timing, inputs, outputs, and context. OpenTelemetry provides the instrumentation layer most teams use for this; the spans it produces are the raw material for trajectory analysis.
Scoring intermediate states requires defining what "correct" looks like at each step, not just at the end. For some steps that is deterministic: a database query either returned the expected schema or it did not. For others it requires a model-graded rubric: did the reasoning step stay within the task scope, did it use the right context, did it produce output consistent with the prior step. Validating agent behavior against expected outcome schemas is where most teams underinvest.
Evaluation gates translate step scores into workflow control. A gate that fires when a step score drops below a threshold can pause the workflow, escalate to a human, or roll back to a checkpoint. Elastic's threat detection system, for example, uses step-level metrics to catch mid-stream reasoning failures before they propagate to final output, with human-in-the-loop gates at critical decision points in their SecOps workflows.
This is also where the gap between offline evaluation and online evaluation becomes structural. A benchmark run is a single trajectory through a controlled environment. Production is thousands of trajectories through conditions the benchmark did not anticipate, across model versions that shift incrementally. The 37% gap between lab benchmark scores and real-world deployment performance documented in enterprise agentic systems is largely a trajectory distribution problem: the benchmark tested one path, production took others.
Connecting evaluation to deployment decisions
Trajectory evaluation is not just a quality tool. It is a deployment signal. If you can measure average trajectory success rate across a representative sample of production-condition inputs, you have a number that predicts whether a workflow will hold at scale, rather than a benchmark score that measures whether it held in a lab.
Detecting quality decay and production drift requires the same infrastructure. When a model update shifts intermediate reasoning distributions, trajectory scores change before final output quality degrades visibly. That is the early warning the Anthropic override-rate data demonstrates at a behavioral level.
At Prefactor, the SDK records every span of an agent's execution, scores each step against activity schemas you define, and maintains a full audit trail of trajectory behavior over time. When a multi-agent orchestrator delegates to a subagent, the trace captures both legs of the handoff and scores the transition, not just the final result. That record is what makes agent evaluation in production auditable rather than anecdotal.
Where to start
Instrument your next agent deployment at the span level before you tune the prompt or swap the model. Score two or three intermediate steps where you know errors tend to originate, and check whether trajectory success rate predicts your current failure modes better than final-output accuracy does. If it does, you have the foundation for a production evaluation gate.
Start evaluating your agents and read through the docs to see how span recording and step scoring are configured.