What this article covers
You will learn how to instrument token flows inside agent spans, separate overhead tokens from productive tokens, and benchmark efficiency across model variants before a workload scales. The Systima analysis published in July 2026 gives a concrete starting point: Claude Code sends roughly 33,000 tokens of system prompt and context overhead before it reads a single word of your prompt, while OpenCode sends around 7,000 tokens for the same task. If your agent runs 10,000 sessions a day, that gap is not a rounding error in your API bill.
The goal here is not to declare a winner between tools. It is to give you the instrumentation approach that makes the comparison possible for your own workload, because the efficient choice depends on what your agent actually does.
Why token overhead compounds faster than you expect
Token pricing is linear at the API layer, but agent costs are not linear in practice. Every agentic loop adds context: tool call results, intermediate reasoning, memory retrieved from a RAG retrieval layer, prior turn summaries. A single user session that takes four tool calls before responding can carry three to five times the tokens of a direct completion.
Goldman Sachs projects a 24-fold increase in enterprise token consumption by 2030, driven by agentic workflows. If your baseline efficiency is already poor, that multiplier arrives on top of waste you have not measured yet.
The Systima finding is useful precisely because it isolates one category of overhead: the fixed cost that the tool incurs before your input even arrives. That 33,000-token figure for Claude Code versus 7,000 for OpenCode is not a measure of output quality. It is a measure of how much the tool spends on its own framing before doing any work for you. At standard API pricing for frontier models in mid-2026, 26,000 extra tokens per session costs roughly $0.07 to $0.26 per call depending on the model tier, which reaches thousands of dollars a month at modest scale.
The broader pattern matters more than the specific numbers: overhead tokens are invisible unless you instrument for them explicitly.
Three token buckets worth measuring
Before you can optimize, you need a taxonomy. When you record spans for an agent session, group token counts into three buckets rather than treating the session total as one number.
System and framework overhead. These are tokens the agent or tool sends regardless of user input: system prompts, tool schemas, memory scaffolding, and any context injected by the framework before the model sees your content. The Claude Code figure from Systima falls here. This bucket tends to be fixed or near-fixed per session, so it hurts most on short tasks.
Retrieval and context tokens. These are tokens pulled in from external sources: documents retrieved by a search step, prior conversation summaries, structured data fetched by a tool. This bucket scales with task complexity and retrieval strategy. A multi-agent orchestrator passing full intermediate results between subagents can push this bucket very high.
Productive tokens. These are the tokens that directly contribute to the answer: the model's reasoning over your specific inputs and its output. This is the bucket you are paying for intentionally. A useful efficiency ratio is productive tokens divided by total tokens. If that ratio is below 0.4, the session is spending more than half its token budget on overhead and retrieval rather than on the actual task.
Measuring these three buckets separately tells you where to intervene. High system overhead suggests the framework or tool is misconfigured. High retrieval overhead suggests your chunking strategy or retrieval scope needs narrowing. Low productive token share on short tasks suggests the fixed costs are dominating.
How to instrument token flows in spans
Every agent framework that follows OpenTelemetry conventions emits spans for LLM calls. The span attributes you need are `llm.usage.prompt_tokens`, `llm.usage.completion_tokens`, and any custom attributes your framework adds for sub-categories. If your framework does not break prompt tokens into sub-categories, you add that classification at the instrumentation layer.
The pattern that works in practice is to record a parent span for the full agent session, then child spans for each LLM call within it. On each child span, you attach:
- Total prompt token count from the API response
- A tag for the call type: system, retrieval, or task
- The tool calls invoked in that step, if any
- The model variant used, because GPT-5.6's three variants carry different pricing and context window sizes, and comparing a mini variant against a full variant without recording which one was used makes your efficiency data meaningless
For coding agents specifically, the overhead profile looks different from customer support agents. A coding agent typically loads file trees, diffs, and linter output into context, so retrieval tokens dominate. A customer support agent loaded with a fixed product knowledge base sees high system overhead per session but relatively flat retrieval overhead. Your instrumentation needs to capture both patterns.
Prefactor records these span attributes and lets you slice session token totals by call type, so you can see how much of a session's spend came from system overhead versus retrieval versus task completion. That breakdown is what makes the Systima comparison actionable for your own deployment, rather than a general observation about someone else's benchmark.
For the OpenTelemetry specifics on span structure, the semantics conventions for LLM spans are evolving, and the GenAI working group has stabilised the core attributes enough that they are usable in production.
Benchmarking efficiency across model variants
OpenAI's release of GPT-5.6 in three variants gives engineering teams a concrete case for why you need variant-level attribution in your spans. A mini variant might cost one-eighth the price per token of the full model. If your agent's task does not require frontier reasoning, running it on the full model means spending the full rate on tokens that the lighter variant handles correctly.
The benchmark approach that surfaces this is straightforward:
- Run the same set of representative tasks through each variant, with full span recording.
- For each run, compute the productive token ratio and the cost per successful task completion.
- Compare not just total cost but cost per unit of output quality, using a scorer that evaluates task completion, not just token count.
That last step is where teams most often skip a step. A model that completes the task in fewer tokens is not automatically more efficient if its success rate is lower. Druva's multi-agent system resolved 68% of support issues without human intervention, saving roughly two hours per complex incident. The efficiency gain there came from task completion rate, not token minimization alone.
The ratio you want is: (cost per session) / (task success rate). A model that costs $0.10 per session at 90% success is more efficient than one that costs $0.06 per session at 50% success, assuming the value of a successful completion is the same.
Benchmarking this ratio across model variants before scaling is the mechanism that prevents discovering the wrong model choice at 100,000 sessions a month instead of 1,000.
The production scaling problem
31% of enterprises now have at least one AI agent in production, and the teams operating those agents are increasingly responsible for their token spend. The FinOps Foundation's 2026 report found that 98% of FinOps practitioners now manage AI spend, up from 31% two years earlier, across roughly $83 billion in annual cloud spend. Token cost is no longer an engineering footnote; it appears in the same budget conversations as compute and storage.
The gap between instrumented and uninstrumented teams becomes visible at scale. Consider what happened with Cox Automotive's 17 agentic solutions: fleet repair estimates that previously took 8 to 48 hours dropped to 30 minutes. At that kind of throughput, a 4x token cost difference between model choices is a six-figure annual line item, not an optimization curiosity.
The same dynamic applies to any agent processing high volumes of similar tasks. Klarna's AI assistant handled 2.3 million conversations in its first month, equivalent to the work of 700 full-time agents. If that system carried 26,000 excess overhead tokens per session, the cost delta across 2.3 million sessions would be material enough to appear in a quarterly review.
This is also why agent evaluation in production needs to include cost metrics alongside quality metrics. An agent that degrades in output quality under load and costs more per session than expected is exhibiting two problems that correlate, and neither is visible without instrumentation.
What to fix when you find overhead
Once you have token flow data, the interventions fall into three categories.
Trim fixed overhead. If your system overhead bucket is large, audit what the framework loads unconditionally. Tool schemas that the agent never calls in most sessions are a common source. For MCP-based integrations, MCP server configuration determines which tool schemas get injected; narrowing the tool set per agent role is often the fastest fix. For some frameworks, you can serve tool schemas on demand rather than loading all of them at session start.
Tighten retrieval scope. If your retrieval bucket is large, the usual cause is overly broad queries returning more chunks than the task needs. Reducing top-k, tightening embedding similarity thresholds, or moving to a reranker that filters before the LLM sees the results all reduce this bucket without touching output quality in most cases.
Match model to task. If your productive token ratio is healthy but your cost per session is still high, the likely cause is running a frontier model on tasks that a smaller variant handles correctly. The three-variant structure of GPT-5.6 is designed to let you route tasks by complexity. The instrumentation you need to do that routing correctly is the same span data that tells you which tasks fail on the lighter variant.
For teams running autonomous background agents, the routing problem is harder because there is no human in the loop to catch a degraded output. You need a scorer on the completion that fires before the result is committed, not a human review step after the fact.
Prefactor attaches quality scores to individual spans and can flag sessions where the productive token ratio falls below a threshold you set, which gives you a signal on sessions worth investigating rather than reviewing all spans manually.
Where to start
Pick one agent, instrument its spans to capture system, retrieval, and task token counts separately, and compute the productive token ratio for 100 representative sessions. That baseline tells you which bucket to address first. Run the same sessions against a cheaper model variant before deciding the frontier model is necessary.
Start evaluating your agents or read the docs to see how to attach token classification to existing spans.