What this guide covers
When a multi-step agent workflow fails, the failure report you receive is usually something like "agent did not complete the task." That tells you nothing actionable. This guide explains why final-output evaluation misses the majority of production failures, which failure modes only show up when you trace individual steps, and how to instrument workflows so you can move from "agent failed" to "step 7 failed because of a tool mismatch at step 4."
The techniques here are relevant whether you are running three agents or thirty, but the urgency scales with the number of steps in your workflows.
Why final scores hide the real problem
A final-output score answers one question: did the last step produce an acceptable result? It does not tell you which upstream decision caused the result to be unacceptable, whether the failure was a one-time event or a pattern, or whether the agent took a reasonable path to a wrong answer or a completely unreasonable path to a right answer.
Research on benchmark versus production gaps shows a 37% performance gap between lab evaluations, which test single-turn tasks, and production deployments, which involve cascading multi-step workflows. That gap exists because the evaluation method does not match the failure mode.
The compounding problem is mechanical. If each agent in a chain succeeds 70% of the time, a three-agent chain succeeds only 34% of the time overall. No single-agent benchmark surfaces that. You only see it when you measure step by step.
A 2025 preprint on multi-agent failures found that 17.14% of failures are step repetitions and 13.98% are reasoning-to-action mismatches, both of which produce plausible-looking outputs that pass final-output checks. The agent arrives somewhere reasonable by a broken route, and you never know.
The failure modes that cascading steps surface
Hallucination attribution
A hallucination in a final response is often not generated at the final step. It enters the context at an earlier step, gets carried forward, and the later steps treat it as ground truth. The AgentHallu benchmark, which analyzed 693 trajectories across seven agent frameworks, established a specific task for this: identifying which step in a multi-step workflow is responsible for a downstream hallucination. The task is challenging even for GPT-5 and Gemini 2.5 Pro, which tells you two things. First, the problem is real. Second, automated attribution alone is not yet reliable enough to replace step-level traces that a human can inspect.
Tool call failures and context drift
A tool call at step 5 can fail because the context passed to it from step 3 was malformed, incomplete, or stale. From the final-output perspective, this looks like "the agent could not complete the action." From a step-level trace, you see that step 3 retrieved a document, step 4 extracted a field that did not exist in that document and silently defaulted to null, and step 5 called the external API with a null parameter.
A healthcare insurance prior authorization workflow deployed on LangGraph saw accuracy rise from 71% to 93% after implementing context isolation between steps. The failures were not model failures. They were state management failures that a final-output check would never catch.
Reasoning divergence across branches
When workflows branch, different execution paths may reason differently about the same underlying facts. Without step-level traces, you cannot tell whether two agents in a pipeline reached the same conclusion by compatible reasoning or whether they contradicted each other internally and one happened to win.
This matters especially for multi-agent orchestrator deployments where a coordinator agent receives outputs from several subagents and synthesizes them. If the synthesis step scores well but the subagent outputs were inconsistent, you have a reliability problem that is invisible at the output layer.
What execution tracing actually gives you
Execution tracing records the inputs, outputs, tool calls, model calls, and intermediate state at every step in a workflow, not just the final result. Combined with step-level scorers, it lets you answer questions like:
- Which step first introduced an incorrect assumption?
- How often does step 4 fail when step 2 calls a specific tool?
- Is reasoning quality degrading across longer workflows because context is growing too large?
Evals and observability are not the same thing. Observability tells you what happened. Evaluation tells you whether what happened was correct. You need both at the step level, not just at the output level.
JPMorgan Chase's COiN platform automates multi-step legal document analysis across parsing, reasoning, regulatory checking, and synthesis. The platform has helped automate 360,000 hours of legal work, but accuracy rates of 85 to 95% mean that errors compound when agents operate autonomously across cascading decisions. At that scale, knowing which step type produces the most errors is the difference between targeted remediation and random tuning.
The cost of not diagnosing at the step level
Silent failures are not free. Organizations that skip structured evaluation spend 3 to 5 times more on incident response than those that implement continuous evaluation. The 3-to-5x figure comes from a Deloitte AI Ops Maturity study and covers direct incident costs; it does not account for the cost of decisions made on incorrect agent outputs before anyone noticed a failure.
TravelJoy switched from one AI agent platform to another for the same knowledge base and the same workflow steps: query parsing, retrieval, grounding check, response generation. The resolution rate went from 24% to 80%. The difference was step-level diagnosis of retrieval failures and grounding mismatches. The final-output scores on the original platform did not surface those failures; someone had to look at the steps.
There is also a token cost dimension. A Concordia University analysis found that agents burn 5 to 30 times more tokens per task than single-turn chatbot calls, driven by excessive retries and context resending at the step level. Those costs are invisible in aggregate spend dashboards. Step-level traces show exactly where the token burn is happening and whether it correlates with failures.
Instrumenting your workflows
The practical starting point is treating each step as a scorable unit. That means:
- Recording the input and output of every step, including tool calls and their responses.
- Applying at least one scorer per step type: a factual grounding scorer for retrieval steps, a schema validator for tool call steps, a reasoning consistency scorer for synthesis steps.
- Correlating step scores with final-output scores to identify which step types predict failure.
Prefactor instruments this at the span level. The SDK wraps each step and records inputs, outputs, tool calls, latency, and model parameters. Step-level scorers then run against each span, and the results are attached to the trace so you can filter by step type, score range, or failure pattern rather than scrolling through raw logs.
Validating agent behavior against expected outcomes is more tractable when you have step-level evidence rather than only a final verdict. You can write activity schemas that specify what a retrieval step should return and catch deviations before they propagate.
The LangChain 2026 State of Agent Engineering survey found that 89% of teams implemented observability for their agents, but only 52% implemented structured evals. The gap between those two numbers is where most step-level failures currently live, observed but not diagnosed.
Detecting quality decay over time also becomes possible once you have step-level baselines. A step that scores 0.91 in week one and 0.74 in week six is a signal worth investigating before it becomes a production incident.
Related reading
- Agent evaluation in production what to measure and how to prove it
- Offline to online evaluation gap agent production failures
Where to start
Pick one workflow that has failed in production without a clear root cause, instrument it with step-level traces, and apply a scorer to the step type most likely to carry failure forward. That gives you a concrete comparison point before you expand coverage.
Start evaluating your agents or read through the docs to see how span-level recording and step scoring are configured.