What you will learn
Benchmark scores tell you whether an agent reached the right answer on a held-out test set. They say almost nothing about how it got there, whether it invented a tool call that does not exist, whether it corrupted its own context three steps in, or whether a reasoning error on step two quietly poisoned every step that followed. This article covers the failure modes that aggregate scoring misses, the metrics that surface them, and how to build an evaluation pipeline that catches quality decay before your users encounter it.
Why benchmark scores are not enough
In April 2026, a benchmark audit found that an automated agent scored 100% on seven of eight leading benchmarks without solving a single task, exploiting weaknesses in evaluation infrastructure rather than demonstrating real capability. That incident is an extreme case, but it illustrates a structural problem: final-answer correctness is the easiest signal to game and the least informative signal about production behaviour.
The gap is quantifiable. Research published in April 2026 found a 37% difference between lab benchmark scores and real-world production performance across deployed agent systems. That gap does not come from model capability. A 2026 Gartner projection estimates that 40% of enterprise AI failures through 2028 will trace to inadequate evaluation and monitoring, not to model limitations.
The reason benchmarks miss production failures is architectural. A benchmark grades the terminal state. An agent running in production takes a trajectory, and a bad trajectory can reach a plausible-looking terminal state. The failure is invisible until it has already caused downstream harm.
The four failure modes benchmarks do not catch
Hallucinated tool calls
An agent can invoke a tool with a schema that does not match the actual API, pass arguments in the wrong order, or call a tool that was removed from the registry. If the downstream system returns a graceful error, the agent may improvise a response that looks correct from the outside. The error never surfaces in a final-answer score.
Sherlocks.ai documented 73 production agent incidents between January and May 2026 and found that tool-call failures are the most common entry point, but they rarely travel alone. Failures stack: a malformed tool call produces a partial response, which the agent treats as complete context, which corrupts the next step. By step four or five, the agent is working from a premise that was never true.
Cascading context corruption
Multi-step agents maintain a working context across steps. If that context is partially overwritten, truncated, or misread at any step, subsequent reasoning is built on a flawed foundation. The final answer may still pass a surface plausibility check. Step-level trajectory evaluation is the only way to isolate where the context broke and what propagated downstream.
Infinite loops and redundant step cycles
Without step-level inspection, a loop looks like latency. An agent that queries the same data source three times and synthesises three identical partial results may still produce a coherent output, but at triple the token cost and with inflated latency that degrades user experience at scale. Measuring token efficiency and cost per task surfaces these cycles; final-answer evaluation does not.
Multi-step reasoning collapse
A reasoning error on step two does not always abort the task. The agent continues, sometimes plausibly, and delivers an answer that looks reasonable but rests on a false intermediate conclusion. This is particularly acute in RAG-based retrieval workflows, where a bad retrieval on one step can bias every synthesis step that follows, and in multi-agent orchestration, where one agent's output becomes another agent's input.
The metrics that actually catch these failures
Aggregate scores measure outcomes. Step-level metrics measure the path. These are the metrics worth instrumenting:
Step-level accuracy. Score each agent action independently: was the tool call valid, was the argument schema correct, did the retrieved context match the query intent. A task-level accuracy of 90% can mask a step-level error rate of 30% if the errors happen to cancel out.
Trajectory faithfulness. Compare the sequence of steps the agent took against the expected sequence for a given task class. Deviations are not always failures, but systematic deviations from expected paths are a signal worth investigating.
Context integrity across steps. Check whether information that was present in context at step N is still accurately represented at step N+2. Corruption shows up as drift between what the agent was told and what it acts as though it was told.
Token-per-step cost and latency. Redundant cycles show up here before they show up anywhere else. A step that costs three times the baseline token count for a routine retrieval is worth examining.
Failure co-occurrence. As the Sherlocks.ai data shows, failures stack. A tool-call error followed by an unusual context size followed by a high token count is a pattern worth flagging even if each individual metric looks borderline.
These metrics require instrumenting agents at the span level, not just logging inputs and outputs at the task boundary. Prefactor records individual spans as agents run, scores each step against configurable quality and risk criteria, and validates the full trajectory against an activity schema you define. That schema is what lets you distinguish a legitimate deviation from an agent going off-script.
Continuous evaluation, not one-off benchmarks
Observability and evaluation are not the same thing. Watching your agents run tells you what happened. Evaluation tells you whether what happened was correct. Most teams that have reached production have the former; a 2026 LangChain survey found that 89% of respondents had implemented observability but only 52% had adopted evals.
The gap matters in practice. Klarna deployed an OpenAI-powered customer service agent that handled 2.3 million conversations in its first month, cutting resolution time from 11 minutes to under 2 minutes. The outcome metrics were strong. The team later pivoted to a hybrid human-AI model after acknowledging the initial approach had limitations. What changed was not the model: it was the evaluation of where the agent needed human review and where it did not.
Goldman Sachs deployed autonomous agents for transaction reconciliation and client onboarding, workflows where a step-level error is not a UX inconvenience but a compliance event. At that fidelity requirement, validating agent behaviour against expected outcomes before and during production is not optional.
Continuous evaluation means running your scorers against live traffic, not just against a static test set. Quality decay and production drift are real phenomena: a model update, a schema change in a downstream API, or a shift in the distribution of user inputs can degrade step-level accuracy without triggering any task-level alert.
Pair that with an audit trail that records every agent action and you have the evidence you need to diagnose regressions, demonstrate compliance, and know exactly where to intervene.
Where to start
Pick one agent that is already running in production, instrument it at the span level, and score three to five steps per task for one week. The pattern of step-level errors will tell you more than any benchmark score. From there, extend coverage to your other agent types and set drift alerts on the metrics that moved.
Start evaluating your agents and read the docs to see how to connect your first agent and define your first activity schema.