What you will find here
This article explains why agents that perform well in testing accumulate small step-level errors in production, how those errors compound into workflow breakdowns that are invisible until they are expensive, and what evaluation practices catch them early. The examples below are drawn from real deployments, with figures attached where they exist.
The benchmark gap
An agent that scores well on a benchmark is not the same as an agent that behaves reliably across thousands of live sessions. Benchmarks measure outcomes on fixed inputs. Production sends variable inputs, at scale, through chains of steps where each decision feeds the next.
According to a November 2025 analysis, 73% of enterprise AI agent deployments experience reliability failures within their first year of production despite passing benchmarks in testing. That number does not mean the agents crash. It means they drift from intended behavior in ways that outcome metrics do not surface.
The Salesforce internal Agentforce deployment is a clear example. After processing more than 1.5 million support requests on their help site, the team found that early iterations were factually accurate but behaviorally wrong. Overly restrictive competitor guardrails caused the agent to refuse legitimate questions, such as a customer asking how to connect Microsoft Teams with Salesforce. The task completed, the refusal looked like a clean exit, but the customer got no answer. Outcome metrics showed zero errors. Step-level review showed a systematic policy mis-application.
How silent failures accumulate
Silent failure is not a single event. It is what happens when a step-level error is small enough to pass without triggering a threshold, but the error recurs or propagates through the rest of the workflow.
Consider a customer support agent that retrieves the wrong product variant from a knowledge base, generates a correct-looking response grounded in that wrong document, and closes the ticket. The outcome: ticket resolved. The reality: the customer received instructions for a different appliance. Fisher and Paykel's Agentforce deployment for appliance support, which increased self-service resolution from 40% to 70%, depended on accurate grounding across thousands of knowledge articles. That accuracy required step-level validation of retrieval, not just measurement of resolution rate. Resolution rate alone tells you what percentage of sessions ended. It does not tell you whether the answer was right.
In multi-step workflows the failure modes are worse. A coding agent that misreads a requirement at step two will produce plausible code that fails downstream tests, or worse, passes them on the wrong behavior. For context on how these agents are typically structured, see how coding agents are categorized and how multi-agent orchestrators compound these risks. When an orchestrator passes bad output from one agent to the next, the downstream agent has no way to know the input was corrupted. Each subsequent step executes with apparent confidence.
Why evaluation is not the same as observability
Watching what an agent does is not the same as scoring whether it did it correctly. The distinction is covered in more depth in our piece on evals versus observability, but the practical consequence is this: you can have complete trace logging and still have no signal on behavioral drift, because drift requires a baseline to measure against.
A 2026 survey of technical leaders found that 80% reported measurable ROI from AI agents, but 40% of agentic projects are projected to be canceled by 2027 due to inability to systematically evaluate deployed agents. The agents are producing value. The teams cannot prove it, or detect when it degrades.
Banner Health's BannerWise deployment illustrates what systematic evaluation enables. The Claude-powered clinical assistant processed more than 1,400 clinical notes for oncology chart summarization. Manual summarization had taken roughly 8 hours per patient across hundreds of pages. 80 to 85% of users reported time savings and improved documentation accuracy. Reaching and sustaining those numbers in a clinical setting required more than a launch metric. It required ongoing validation that each summarization step was meeting the accuracy threshold the clinical team had set, not just that the output existed.
This is what step-level evaluation addresses: each discrete action in an agent's run, scored against a defined schema of expected behavior, with the score recorded alongside the trace. The silent failure gap between pre-production and live behavior is where most production incidents originate.
What step-level evaluation actually catches
Step-level evaluation catches four categories of failure that outcome metrics miss:
- Retrieval mis-grounding: the agent used the right tool but fetched the wrong document or record. The output looks grounded. The ground is wrong.
- Policy drift: the agent's guardrail application changes over time as context shifts, without any change to the system prompt. What was refused last month is allowed today, or vice versa.
- Compounding error propagation: a small mis-classification at step two changes the branch the agent takes at step four, producing a plausible but incorrect final output.
- Latency and cost exceedance: individual steps consuming more tokens or time than the workflow budget allows, which compounds across sessions into significant cost drift. This is explored in more detail in measuring hidden token overhead and agent efficiency.
PwC's Claude-based agent deployments for finance, supply chain, and deal execution are running in production with delivery improvements up to 70% reported by clients. At that scale, across journal entry agents, variance analysis agents, and RFP agents, the cost of a single policy mis-application or retrieval error is not contained to one session. It propagates across the process.
Prefactor addresses this by recording spans at the step level, scoring each span against an activity schema defined by the team, and flagging deviation in real time. The audit trail attaches to each run, so when a failure is discovered downstream, the step that originated it is recoverable. For teams working across internal operations agents or RAG retrieval agents, that traceability is what makes remediation tractable rather than speculative.
If you want a structured framework for what to measure and how to report it, agent evaluation in production covers the metrics and their limits. For teams concerned with behavioral drift specifically, detecting agent quality decay and production drift walks through the monitoring patterns in detail.
Where to start
Define what correct behavior looks like at each step of your agent's workflow before you instrument it. Without a schema, evaluation produces scores with no reference point. Once you have that schema, attach scoring to the spans you are already collecting.
Start evaluating your agents or read the docs to see how activity schemas and step-level scoring are configured.