What final-output evaluation cannot see
A benchmark measures whether an agent reached the right answer. It does not measure whether the agent reached it correctly. For single-step tasks, that distinction rarely matters. For multi-step agents operating in production, it is the difference between catching a failure before it causes harm and discovering it in an incident report three weeks later.
The pattern works like this: an early step produces output that is wrong but plausible enough that the next step accepts it without question. That step builds on the corrupted context, adds its own work, and passes the result forward. By the final step, the error is load-bearing. The agent produces an output that looks coherent, passes quality checks on its surface, and may even score well on automated evaluation, because automated evaluation is reading the final answer, not the trajectory that produced it.
This is not a theoretical risk. Carnegie Mellon benchmarks found that leading AI agents complete only 30 to 35 percent of multi-step tasks reliably in production conditions. The gap between benchmark performance and production reality is documented and consistent.
How cascading failures differ from single-step hallucinations
A single-step hallucination is recoverable. The agent says something wrong, a reviewer catches it, the output is discarded. The scope is bounded by the step.
A cascading failure is different in kind, not just degree. The error propagates through the system, acquiring legitimacy at each step because downstream components treat upstream output as ground truth. By the time the failure surfaces, isolating the origin requires tracing every intermediate state, not just reading the final output.
The Cursor support agent incident in April 2025 illustrates the mechanics clearly. The retrieval step returned no results for a live query. Rather than halting or flagging the empty retrieval, the synthesis step filled the gap by fabricating a device login policy. That fabrication passed through to customers as if it were documented fact. The retrieval failure was invisible to any evaluation that only read the final response, because the final response was grammatically correct, confidently stated, and topically relevant. It was simply false.
The same propagation pattern appeared in Recursion Pharmaceuticals' FDA filing, where an agent submitted a regulatory document containing three fabricated citations. Each citation was structurally valid, formatted correctly, and contextually plausible. The fabrications only became apparent when reviewers checked the sources directly. No final-output quality score would have caught them.
Data from production deployments shows that tool misuse and incorrect tool arguments account for 31% of production agent failures, making it the most common proximate cause in 2024 to 2025 deployments. These failures originate at the tool-call step, not the response step. An evaluation architecture that only scores the response step cannot see them.
Why benchmarks are structurally blind to this
Benchmark design optimises for measurability. Final outputs are measurable: you can compare them to a reference answer, run a classifier over them, or have a human rate them. Intermediate steps are harder to evaluate because there is no clean reference for what the internal state of an agent should look like at step three of a seven-step workflow.
This creates a gap that offline evaluation cannot close. Benchmark environments also control inputs more tightly than production does. An agent tested against a curated dataset will rarely encounter the retrieval failure that the Cursor agent encountered, or the state corruption that delayed treatment for seven patients across three NHS London hospitals in February 2026 due to multi-step workflow state corruption.
The NHS incident is worth pausing on. The agent was processing triage workflows. The state corruption that caused delays was not a failure of the final output in isolation: it was an error in how intermediate workflow state was maintained and passed between steps. Any evaluation that only read the triage output would have seen a completed task.
What a step-level evaluation architecture looks like
Step-level evaluation adds a scoring layer at each stage boundary rather than only at the terminal output. The practical components are:
- Span instrumentation. Every tool call, retrieval result, and intermediate generation is recorded as a distinct span with its inputs, outputs, and execution metadata. This is the prerequisite for everything else.
- Per-step quality scoring. Each span is scored independently, against criteria appropriate to what that step is supposed to produce. A retrieval step is scored on whether it returned relevant, non-empty results. A synthesis step is scored on whether its output is grounded in what the retrieval step actually returned.
- Schema validation at stage boundaries. Before the output of one step is accepted as input by the next, it is validated against a declared schema for that handoff. An empty retrieval result that the synthesis step would otherwise silently accept instead raises a flag.
- Trajectory-level review. The sequence of spans is inspected for patterns that indicate process failure rather than output failure: step repetitions, reasoning-to-action mismatches, and tool calls with arguments that contradict earlier reasoning.
Trajectory validation at stage boundaries catches 67% of failures that single-output evaluation misses. That figure comes with a scope condition: it applies to multi-step agents with at least three sequential tool-use stages. Single-step agents do not benefit in the same way.
Prefactor instruments this by recording each span through the SDK, applying configurable quality and risk scores per step, and validating each span against activity schemas before the trace is closed. The audit trail captures intermediate states, not just the final answer. For autonomous background agents and multi-agent orchestrators, where human review of every output is not practical, that trace is the primary mechanism for post-incident diagnosis.
When step-level evaluation becomes mandatory
For low-stakes, reversible tasks with short chains, final-output evaluation is a reasonable starting point. The cost of a step-level architecture is non-trivial: it requires instrumentation, schema design, and additional scoring compute.
The calculus changes when any of the following apply:
- The agent operates in a regulated context where intermediate decisions require an audit trail. The Morgan Stanley trading agent incident in January 2026, which resulted in $47 million in unauthorized trades, was contained by an overnight monitoring team, not by evaluation infrastructure. Step-level controls with circuit breakers on tool-call velocity would have been a faster detection path.
- Actions taken by the agent are irreversible or externally visible before review is possible. Database deletions, like the PocketOS incident that caused a 30-hour outage in April 2026, are the clearest case. The step that issued the deletion command was the control point. The final-output evaluation, which read a completed task, saw nothing wrong.
- The agent operates across more than three sequential steps where intermediate outputs are trusted by downstream components without independent verification.
For more detail on where evaluation typically falls short before go-live, see evaluation passing failures in production customer-facing agents and silent failures in production agents.
Step repetitions cause 17% of multi-agent failures and reasoning-to-action mismatches cause 14%, both undetected by final-output evaluation. These are not edge cases. They are systematic failure patterns that appear in production at meaningful rates and that only become visible when the trajectory is recorded and inspected.
The Zillow rental inquiry agent that generated 3,400 responses containing housing discrimination violations in February 2026 was not producing isolated hallucinations. It was consistently applying a corrupted reasoning pattern across a large volume of requests. A step-level scorer applied to the demographic-proxy selection step would have flagged the pattern far earlier than any review of final responses, which were each individually plausible.
Where to start
Instrument your agent's spans first, so you have a trajectory to evaluate. Add per-step schema validation at the handoffs where corrupted output would be most consequential. Then layer in quality and risk scoring per step, starting with retrieval and tool-call stages.
Start evaluating your agents and review the docs for instrumentation and schema configuration.
