Step-level accuracy compounds. If each step in a ten-step agent workflow succeeds 85% of the time and each step depends on the one before it, the probability that all ten succeed is 0.85 to the power of ten, which is roughly 20%. Your dashboard shows a healthy agent. Four out of five runs contain at least one failed step. The gap between those two numbers is where most production incidents live, and final-output evaluation cannot see it.
Why per-step accuracy is the wrong number
Per-step accuracy measures the probability that a single step succeeds. That feels like the right signal, and it is not, because steps are not independent events. Each step's output becomes part of the context for the next, so a misclassification at step two shapes the retrieval query at step three, the tool call at step four, and the answer at step five.
The arithmetic scales badly. At 95% per-step accuracy, a ten-step workflow still finishes correctly only about 60% of the time. At 90%, about 35%. At 85%, about 20%. A five-step workflow at 85% lands at 44%. The effect is measurable at three or four steps and operationally significant by five, and most production agent workflows involve more than five steps.
Final-output scoring hides this because it averages across tasks. Many partial failures still produce a plausible-looking answer. An agent can reach the correct output through a wrong path, which means it got lucky rather than competent, and on the next run the same wrong path produces a wrong answer.
The same compounding shows up across repeated runs, not just across steps. Sierra's tau-bench found that agents succeeding on 60% of single runs fell to 25% when required to complete eight consecutive runs of the same task. A single passing evaluation tells you the agent can succeed; it does not tell you it succeeds reliably enough to deploy.
The failures that pass a final-output check
Research on multi-agent failure modes found that 17.14% of agent failures are step repetitions and 13.98% are mismatches between the agent's stated reasoning and the action it actually took. Together that is roughly 31% of failures that output-only evaluation will not surface, because the terminal output still looks plausible.
Three patterns account for most of what goes wrong between step one and the final answer.
Retrieval returns nothing useful. Cursor's customer support agent fabricated a device login policy in April 2025 because the retrieval step returned empty results from live traffic. Staging tests ran against a complete knowledge base, so the empty-retrieval case never appeared. The model filled the gap and the fabrication propagated through every downstream step unchanged. Only step-level inspection would have flagged the retrieval failure.
Tool selection drifts. Agents choose the wrong tool for an intermediate step, or the right tool with malformed arguments, and the answer still looks coherent because the model compensates from prior knowledge. This is especially common after model updates: Scale AI's 2025 report found 34% of enterprises experienced unexpected agent behaviour changes after a model update, with 12% experiencing production incidents severe enough to need human intervention. A step that previously returned a plain integer now returns a formatted string, and the chain breaks at a point that has nothing to do with the model's capability.
Reasoning degrades on corrupted context. A step that needs context from three earlier steps fails silently if any of them produced low-confidence or off-target output. Analysis of 847 enterprise agent deployments found that 60% of production failures trace to data quality, context gaps or process problems visible at intermediate steps, not to model limitations.
Add the repetition lock: the agent hits an obstacle, fails to resolve it, and re-executes the same step. Customer support and voice agents are especially exposed because the conversation context grows with each retry, so token cost rises and response quality drops at the same time.
What production teams measured, and what changed
The teams that have moved agents to production at scale converged on the same move: score the trajectory, not the endpoint.
Motorway built a production evaluation pipeline in July 2026 that tracks tool selection accuracy, task completion rate and query correctness as separate metrics. Before it, the dealer stock search agent returned incorrect results in roughly one query in eight. After instrumenting the full trajectory, tool selection accuracy rose from 87% to 98%, task completion from 82% to 96%, and issue detection time dropped from hours to minutes. A 70% completion rate had looked acceptable until the tool selection data showed 30% of completed tasks had used a fallback tool: technically successful, methodologically wrong.
An enterprise case documented by Insight Global in March 2026 showed tool-call accuracy improving from 60% to 96% and self-correction rate from 15% to 75% after the team shifted from output validation to trajectory validation. Final-output pass rates had been masking the churn underneath.
MyOperator, running more than 100 voice agents across millions of monthly calls, improved its hallucination score from 2.8 to 9.5 out of 10 after per-step instrumentation identified the specific steps where agents invented product features. The model did not change. Knowing which steps to fix did.
Uber runs roughly 60,000 agent task executions a week through its MCP gateway and built an Agent Studio to visualise execution traces and step-level failures; by early 2026, 84% of Uber developers used agentic coding tools daily, with checkpoint-based recovery catching trajectory deviations before they reached downstream systems. LinkedIn structured its candidate sourcing system as a hierarchical graph that breaks work into the smallest possible tasks, which is an evaluation strategy as much as an architecture: shorter trajectories have fewer compounding steps. Discord cut its deployment cycle from four weeks to one by catching step-level issues in pre-release simulation rather than in production.
Anthropic's operational data adds a leading indicator: when human override rates climbed from 5% to 12% over two weeks, system-level quality incidents followed within a week. A human choosing to intervene mid-workflow is responding to the reasoning path, not the final output.
Three ways to score a trajectory
Per-turn rubrics
Score each step against a defined expected output, independently of the final answer: the right tool was selected, the output matches the expected schema, the reasoning references the correct prior context. Each step gets pass, fail or partial. Aggregate and you get a distribution of failure points across the workflow rather than one endpoint score. For RAG retrieval agents, the rubric should ask whether the retrieved chunks were relevant and whether the model cited them rather than substituting prior knowledge; that catches the Cursor pattern before the hallucination reaches the next step.
Tool-call validation
Check structurally that every external call matches expected behaviour: correct tool, valid arguments, expected response schema, correct handling of error responses. No human-readable judgment is needed. Coding agents and internal ops agents have the highest exposure here because their workflows chain many external systems.
Reasoning sequence audits
Compare what the agent said it would do with what it did, step by step. This is the most demanding of the three because it needs a scoring model or human review, but it is the only method that catches a reasoning-action mismatch. Morgan Stanley's DevGen.AI evaluated a multi-step pipeline across parsing, contextual understanding and specification generation rather than just final translations, and processed 9 million lines of legacy code by June 2025. A flawed contextual understanding step would have produced wrong specifications regardless of how well the final step performed.
The metrics these produce per run are narrow and specific: fraction of tool calls that were unique within the session (a proxy for repetition), reasoning-to-action match rate, intermediate outputs within schema constraints, and whether each self-correction was substantiated by new information or was an arbitrary retry.
What you need in place
None of this works without execution traces that capture inputs and outputs at every step boundary, not just at task entry and exit. If your tracing records the first prompt and the final response, you are scoring the last mile of a relay and ignoring the first nine. OpenTelemetry is the instrumentation layer most teams use; the spans it emits are the raw material.
Prefactor records each step as a named span with its inputs, outputs, tool calls and latency, scores those spans against per-turn rubrics, and validates them against activity schemas that define expected behaviour. A failure is attributed to the step, not folded into a workflow-level score, which is what makes root cause analysis tractable in production and lets you detect quality decay at the step where it starts.
Two limits worth stating. Trajectory evaluation does not stop mattering at high per-step accuracy; it shifts from a triage tool to a monitoring tool, catching tail cases and model-drift signals that aggregates miss. And it does not require an architecture rebuild: if your framework already emits structured spans, you are adding scoring and schema validation on top of existing telemetry.
For multi-agent orchestrators, one agent's step output is the next agent's step input, so evaluating each trajectory independently is necessary but not sufficient. The handoff payload needs validating at the boundary too. How to contain that propagation once it starts is covered in how cascading failure compounds across multi-step agent workflows.
Where to start
Pick one workflow already in production and pull its execution traces for the last 500 runs. Apply a per-turn rubric to each step and map where the failure rate is highest. Then run the same workflow eight consecutive times and compare single-run success to consistency. Those two numbers will tell you more about the agent than any benchmark score you have seen.
Start evaluating your agents or read the docs to see how span recording and rubric scoring connect to your existing tracing.