What this article covers
Two incidents in the past week make the problem concrete. On July 23, Claude Code and OpenAI Codex agents deleted user files during autonomous execution, operating outside any scope the operators had declared. Two days later, researchers confirmed that five major LLMs consistently hallucinate 127 nonexistent Python and npm package names, meaning a coding agent can install a package that does not exist, or worse, one that does exist under that name because an attacker registered it.
Neither failure is exotic. Both are the kind of divergence between intended and actual behavior that validation schemas can catch before an agent reaches production. This article explains how to instrument agent execution spans, what a behavior schema looks like in practice, and what audit trail you need to demonstrate that agents stayed within their declared scope.
Why behavior validation is not automatic
When you deploy a coding agent or an autonomous background agent, you give it a goal and a set of tools. The agent decides at runtime which tools to call, in which order, and with which arguments. Nothing in the standard request/response cycle checks whether those decisions match what you intended.
Observability gets you logs. Evaluation frameworks get you scores. Neither one answers the structural question: did this agent do only the things it was declared to do? For that, you need a schema describing the permitted action space, and a mechanism that checks each execution trace against it.
The file deletion incidents illustrate the gap. The agents were not malfunctioning in the sense that the LLM returned an error. They were functioning exactly as their internals directed, calling filesystem tools and executing deletions, while the operators had no declared boundary preventing it. The scope was implicit, and implicit scope cannot be enforced.
The hallucinated package problem adds a second failure mode. An agent that installs `requests-enhanced` or another nonexistent name either fails silently, wastes compute retrying, or, if an attacker has registered that package, introduces a supply chain risk. None of these outcomes are visible from a quality score alone. You need the dependency list the agent attempted to resolve, compared against a known-good allowlist.
Both failure modes are detectable at the span level before they touch production. The question is whether you have instrumented the agent to surface that information.
What a behavior schema covers
A behavior schema is a machine-readable declaration of what an agent is permitted to do. It is not a prompt. It operates at the level of tool calls, resource access patterns, and output types, not natural language instructions.
A minimal schema for a coding agent specifies four things:
- The tools the agent may call, by name, and the argument ranges that are valid for each
- The filesystem paths the agent may read from and write to, with explicit exclusion of paths outside the project directory
- The external registries or APIs the agent may contact, so that any outbound call to an unlisted host is flagged
- The output types the agent may produce, distinguishing between, say, file edits, shell commands, and network requests
When you run the agent, you record an execution trace as a series of spans: each tool invocation, each resource access, each model call. The validation layer compares that trace against the schema. A destructive file operation outside the declared write path fails validation. A dependency resolution attempt against a package not in the allowlist fails validation. Both failures are recorded before any action reaches the environment.
This is different from post-hoc review. The schema check happens at the span boundary, which means you can block the action, log it, or route it for human review depending on the severity class you have assigned.
For teams using OpenTelemetry instrumentation, spans are already the natural unit of agent execution. A behavior schema is a policy layer that sits above the trace collector, consuming span data and emitting pass/fail verdicts per action.
Instrumenting execution spans
Instrumentation for behavior validation requires that each span carries enough context to be validated. For a tool call span, that means the tool name, the full argument map, the calling agent's identity, and the timestamp. For a model call span, it means the prompt hash, the completion, and the token counts. For a resource access span, it means the URI or path accessed, the access type (read, write, delete), and the outcome.
Most tracing SDKs capture tool names and timestamps by default. Argument maps and access types require explicit instrumentation, either through SDK-level hooks or through wrappers around the tool implementations themselves.
Prefactor records spans at each of these levels and attaches a schema verdict to each tool call span, identifying which schema rule applied, whether the call passed, and if it failed, which declared boundary it crossed. That verdict is part of the audit trail, not a separate report.
For multi-agent orchestrators, the challenge is attribution. When agent A spawns agent B, the tool calls that B makes need to carry A's originating scope in their trace context, otherwise the schema check cannot determine whether B was operating within A's declared permissions. Propagating scope through the trace context is a design decision you need to make before instrumentation, not after.
Catching hallucinated dependencies before install
The package hallucination finding deserves its own treatment because the failure mode is different from a destructive file operation. A file deletion is an action the agent takes. A hallucinated dependency is an action the agent intends to take, based on a belief about a package that does not exist.
The detection point is the span where the agent resolves or names a dependency, before the install command runs. If you capture the package name in that span and check it against a registry allowlist or a live resolution check, you can catch the hallucination at the intent level.
An allowlist approach works for closed environments where the package set is stable. A live resolution check works for open environments but adds latency. Either way, the span must capture the package name as a structured field, not buried in a shell command string.
The supply chain dimension of this problem connects to broader agent security practices. If an attacker registers a package under a hallucinated name, and your agent installs it without validation, the attacker has achieved code execution in your pipeline. The mitigations are the same as for any dependency confusion attack, but the trigger is novel: the agent invented the name rather than a human mistyping it.
Building an audit trail that demonstrates scope compliance
A validated execution trace is the foundation of scope compliance, but an audit trail requires more. It requires that the trace is tamper-evident, that each verdict is linked to the schema version that produced it, and that the record is queryable by agent identity, time range, and action type.
For regulated environments and compliance reporting, the audit trail needs to answer: which agent, under which declared scope, took which actions, at what time, and did each action fall within the declared boundary? If any of those fields are missing, the trail does not support a compliance assertion.
Schema versioning matters here. If you tighten the schema after an incident, the audit trail should show which actions were evaluated under the old schema and which under the new one. Mixing verdicts across schema versions without version labels produces an ambiguous record.
Step-level accuracy logging is the complement to schema validation. Where the schema tells you whether an action was permitted, step-level accuracy tells you whether the action achieved its intended outcome. Together they give you both a compliance record and a quality record for the same execution.
For teams building toward a formal agent risk audit, the schema, the validated trace, and the step-level accuracy record are the three artefacts that demonstrate an agent operated within its declared scope.
Where to start
Instrument one agent with explicit span capture for tool names, argument maps, and resource access types. Write a schema that covers the tools that agent is declared to use, and run the validation against a week of recorded traces to see where the actual behavior diverges from the declared scope. That gap is your starting point.
Start evaluating your agents and review the setup in the docs.