Token efficiency as a window into agent behavior
When an agent sends 33,000 tokens of overhead before it has read a single word of your prompt, the immediate reaction is to reach for the billing dashboard. That instinct is reasonable, but it misses the more useful signal: an agent that consumes that much context before beginning work is one whose internal state you cannot predict. Token overhead, and especially its variance across invocations, tells you something about behavioral consistency that output quality scores often cannot.
This article is about using token efficiency as a reliability signal in agent evaluation, alongside quality and risk. If you want the cost and efficiency framing, the companion piece measuring what agents actually cost covers that ground.
What the overhead numbers actually reveal
In July 2026, Systima published a controlled analysis of Claude Code's token consumption. The finding that attracted attention was that the agent sends approximately 33,000 tokens of system overhead before it processes the user prompt. The full analysis is here.
The number is large, but the number alone is not the reliability problem. The reliability problem is what that overhead consists of and whether it is consistent. Overhead that is large but stable means your agent is doing the same things in the same order every time it starts. Overhead that spikes, compresses, or shifts between invocations means the agent's pre-task state is changing in ways you have not authorised and may not be able to explain.
Consider what happens when overhead grows by 40% between two otherwise identical invocations on the same task type. The agent may be loading different tool definitions, expanding its context with prior conversation fragments, or resolving ambiguity in the prompt by fetching additional grounding material. Each of those is a distinct behavioral path, and any of them can produce different outputs even when the input looks the same. Output quality scoring catches the symptom; overhead variance catches it earlier.
Research published in November 2025 found that agent performance drops from 60% on a single run to 25% across eight consistent runs, with cost variations of up to 50x for similar accuracy levels. That gap between single-run and multi-run performance is, in part, a token efficiency story. Agents that expand their context differently on each invocation are agents that are taking different internal routes to the same stated goal.
Variance as an early-warning indicator
Reliability in production is not measured by whether an agent succeeds on a given task. It is measured by whether the agent behaves consistently enough that you can reason about it across thousands of invocations. The gap between benchmarks and production reliability is precisely here: benchmark runs are typically single invocations under controlled conditions, and they tell you nothing about variance.
Token overhead variance gives you a signal that is earlier and cheaper to collect than most quality metrics. You do not need to evaluate the output to know that something changed in how the agent prepared to produce it. A monitoring system that tracks the distribution of overhead tokens per agent, per task type, and per model version can surface instability before the instability reaches the output layer.
Specifically, the patterns worth watching are:
- Baseline drift: the median overhead for a given task type rises steadily over days or weeks, without a corresponding change in task complexity. This often indicates prompt template growth, accumulated context from prior sessions, or tool registry expansion.
- Spike events: individual invocations produce overhead two or three standard deviations above the baseline. These are worth inspecting individually, because they frequently correspond to unusual tool calls, unexpected retrieval paths, or fallback behaviours that only activate under specific conditions.
- Compression events: overhead drops sharply. This sounds like good news but is often not, because it can indicate that context the agent normally uses is failing to load, producing a leaner but less-informed execution.
Recent benchmarking work published in February 2026 found that reliability has improved at roughly half the rate of accuracy for general agent benchmarks, which means the industry's measurement focus has been on output quality at the expense of behavioral consistency. Token variance monitoring is one way to rebalance that.
How this fits into a reliability-scoring practice
Token efficiency variance should sit alongside, not replace, output quality and risk scoring in any agent evaluation practice. The three signals cover different failure modes. Quality scoring catches outputs that are wrong or harmful. Risk scoring catches actions that exceed the agent's authorised scope. Overhead variance catches behavioral instability that has not yet produced a bad output but is trending toward one.
Druva's DruAI system, deployed via Amazon Bedrock AgentCore for cybersecurity incident response, resolved 68% of support issues without human intervention and cut time-to-resolution by 58%. At that scale, with thousands of customers depending on consistent agent behaviour, overhead variance in the context-loading phase would be the kind of signal that precedes degraded resolution rates, often by hours.
Anthropic's own internal use of Claude as a coding agent for production bug fixes, which shipped over 800 individual fixes and reduced error rates by a factor of 1,000, required exactly this kind of behavioral consistency. Eighty percent of new production code authored by an autonomous agent is only tenable if the agent's execution path is predictable enough to trust at that volume.
For teams managing multi-agent orchestrators or autonomous background agents, the concern compounds. Overhead variance in a sub-agent propagates into the orchestrator's context, which means one unstable agent can shift the token distribution of every agent downstream in the pipeline.
What to collect and where to surface it
The practical requirement is span-level token recording for every agent invocation, broken down by phase: system prompt, tool definitions, retrieved context, conversation history, and user input. Without that breakdown, you can see that overhead changed but not why.
Multiple cost monitoring tools launched in July 2026 specifically for coding agents, most focusing on spend alerting. Spend alerting is useful, but it fires after the fact. Variance alerting on token distributions per phase fires before the bill arrives and before the output quality degrades, because behavioral instability typically precedes output degradation.
Prefactor records per-span token counts as part of its agent evaluation instrumentation, which means overhead variance is available as an input to reliability scoring alongside quality and risk signals. The audit trail captures what changed between invocations, so when a spike event occurs you have the context to diagnose it rather than just the alert.
For teams building evaluation practices from scratch, the relevant questions to ask about any monitoring setup are: does it record token counts by phase, does it surface variance over time rather than point-in-time totals, and does it connect overhead signals to downstream quality outcomes so you can validate whether variance actually predicts degradation in your agents.
Where to start
Choose one agent in production that runs frequently enough to build a distribution, at least fifty invocations per week. Start recording overhead broken down by phase for every invocation for two weeks. Look for baseline drift and spike events before you look at output quality. That sequence is more diagnostic.
Start evaluating your agents and see the Prefactor docs for how to instrument spans and surface token variance alongside quality scoring.