What trajectory blindness costs you
Your agent passes every benchmark. It scores well on final-output correctness. Then in production it fabricates a case citation, loops through the same tool call four times, or hands a downstream agent corrupted data that compounds through six more steps. None of that registered during evaluation because you never looked at the steps in between.
This article maps the failure types that live inside multi-step agent trajectories, explains why they survive final-output checks, and describes how to instrument agents to catch them before they compound. The target is engineers building or operating multi-step agents in production, not researchers optimizing benchmark scores.
Why benchmark scores stop predicting production quality
Benchmarks measure whether the final answer is correct. For single-step tasks that is often enough. For multi-step agents it is structurally insufficient, because an agent can reach a correct final answer through incorrect intermediate reasoning, and it can produce a confident, well-formatted wrong answer after a failure that happened five steps earlier.
Stanford and OpenAI research on reasoning models illustrates the gap clearly. OpenAI o1 achieves 93% on AIME math when re-ranking 1,000 samples with a learned scoring function, but only 74% with a single sample. The differentiator is step-level confidence estimation, not final-answer quality. The model reaches wrong answers through plausible-looking intermediate steps that a final-output check never scrutinizes.
The same dynamic appears in coding agents. Research on SWE-Bench Verified found that on harder multi-file tasks, agents fail on planning decomposition and instruction interpretation, not final code correctness, and top models drop 15 to 19 percentage points in accuracy when test suites are adversarially strengthened. The failures are upstream: misscoping which files to touch, misreading the problem requirements. The code the agent writes is internally consistent with its wrong plan.
This is the core of the gap between offline evaluation and production agent failures: offline tests check outputs, production exposes trajectories.
A taxonomy of step-level failures
Step repetitions and reasoning-to-action mismatches
A 2025 arXiv analysis of multi-agent failures, cited by Augment Code in June 2026, found that 17.14% of failures are step repetitions and 13.98% are mismatches between reasoning and action. Both categories share the same detection problem: the agent produces a coherent, confident output while doing the wrong thing internally.
Step repetitions occur when an agent re-executes a tool call or reasoning step it already completed, usually because it failed to update its working memory after the previous execution. The final output may still look correct if the repeated step happens to be idempotent. If it is not idempotent, you get duplicate writes, double charges, or conflicting state, and the agent reports success.
Reasoning-to-action mismatches are subtler. The agent's stated reasoning describes one action, but the actual tool call or output reflects a different one. This surfaces in coding agents where the model narrates editing file A and then edits file B, or in retrieval agents where the stated query differs from the query sent to the index.
Planning defects and scope creep
Planning failures happen at the decomposition stage, before any tool is called. The agent misinterprets the task scope, skips a required sub-goal, or sequences steps in an order that makes later steps impossible to complete correctly.
Enterprise deployment analysis covering 2024 and 2025 found that scope creep and data quality issues account for 61% of all AI agent failures. The mechanism is trajectory corruption: the agent drifts toward confident hallucinations because no intermediate validation catches the point where its plan diverged from the task. Legal agents hallucinating case citations and customer service agents fabricating policies both pass final-output confidence metrics until a human escalates the issue in production.
This is why detecting agent drift through activity schema validation needs to happen at step boundaries, not just at the end of a session.
Tool misuse cascades
Tool misuse cascades are the failure mode where a single wrong decision early in a chain corrupts every downstream step. The Vector Institute analyzed a multi-agent system for Airbnb built on a Council Analytics architecture and found that a single wrong tool call early in the chain corrupts every downstream step across planner, database, and Python agents. The final-output check sees a confident answer. The trajectory shows a propagating error that no single agent in the chain detected.
This is cascade amplification: the cost of a step-one failure is not one wrong step, it is every step that consumed that step's output. In a six-step pipeline, a tool misuse at step two corrupts steps three through six. Detecting it at step two costs a single retry. Detecting it at step six costs a full re-run, plus whatever side effects the intermediate steps produced.
The multi-step failure attribution problem is precisely this: tracing a wrong final output back to the step that originated it, when every intermediate step looks locally plausible.
Information flow corruption in high-stakes domains
Clinical AI research comparing single-agent and multi-agent architectures on mental health screening found that single-agent prompting produced unsupported severity escalations in 33.2% of cases, emotional reasoning drift in 27.6%, and inconsistent diagnostic logic in 21.3%, while multi-agent systems with audit agents intercepted cascading failures before prediction generation. Trajectory validation at stage boundaries caught 67% of failures that single-output evaluation missed.
The structural insight is that workflow boundaries are natural inspection points. When perception, retrieval, and inference are separate stages, you can validate the output of each before it becomes the input of the next. A flat single-agent architecture collapses those boundaries and eliminates the inspection surface.
What instrumentation actually catches
Catching step-level failures requires recording what happens at each step, not just the final output. That means spans for every tool call, the actual arguments sent and the actual response received, the agent's stated reasoning before the call, and the state of working memory before and after.
Amazon moved in this direction with the March 2026 general availability of AgentCore Evaluations, which inspects individual tool calls and reasoning steps alongside final outputs, with built-in evaluators for tool selection accuracy and goal success rate. The shift from final-answer-only to continuous step-level monitoring reflects the production reality that trajectory failures are the common case, not edge cases.
At Prefactor, the SDK instruments agents at the span level, recording each tool call, the reasoning that preceded it, and the arguments as sent. Scores for tool selection accuracy and step-sequence validity run against those spans, not against the final output. An activity schema defines what the trajectory is supposed to look like, and deviations surface as findings before they compound into production incidents.
For engineers building multi-agent orchestrators or RAG retrieval agents, the practical starting point is instrumenting handoff points: what did agent A send to agent B, and does that match what agent A said it was sending. That single check catches both reasoning-to-action mismatches and information flow corruption at the boundary where they are cheapest to catch.
You can find more detail on what step-level metrics to collect and why in the guide to step-level evaluation and measuring agent actions before failure compounds.
Where to start
Pick one multi-step agent in production and instrument its handoff points first: the arguments each tool call actually sends and the reasoning the agent stated before sending them. That surface catches the two most common failure categories without requiring a full evaluation redesign.
Start evaluating your agents or read the docs to see how span-level recording maps onto your existing agent framework.