Problems · in production

An agent that can read everything can repeat it anywhere.

An agent credentialed to read a system can repeat what it read anywhere, and no amount of prompt review bounds where.

run-record · scope checkexample
Illustrative run, the moment a scope check catches a repeat
Agent scopesupport tickets only
Run 4,182 · readcustomer billing record
Output draft containscard digits, home address
Actionheld before send
"What did it touch?"one query, per run
every run records what the agent accessed and what left with it
§01 / THE SYMPTOMyou see: the signals
TL;DR

An agent can repeat anything it can read. Record what every run touched, check scope as it happens, and hold an action that moves data outside the agent's job before the output leaves.

The symptom

What unbounded exposure looks like

Exposure rarely announces itself. These are the shapes it takes in a fleet nobody watches for scope.

01
One run repeats what another read

The same agent on the same task takes a different path each run. An agent that read a customer record on Monday can surface it in an unrelated answer on Thursday, and reading the prompt tells you nothing about which run will.

02
PII rides along in the prompt

User uploads and customer records flow into prompts because that is how the agent does its job. From there they reach models, tools, and traces that nobody listed when access was granted.

03
Shadow tools, shadow agents

Staff paste company data into AI tools nobody approved, and unregistered agents move the same data on a schedule. Neither shows up in any inventory you hold.

04
Every deployment stalls at security review

The security team holds veto power over each agent, and their first question is what data it can leak. Without a per-run answer, the honest reply is a shrug, so the launch waits.

§02 / WHY IT HAPPENScause: not carelessness
Why it happens

Why good teams end up exposed

Each is a reasonable trade under deadline. Together they make exposure unbounded and unwatched.

01
Non-determinism breaks the boundary

With deterministic software, a code review bounds what data can go where. An agent chooses its own path each run, so the only real boundary is a check on what each run actually did.

02
Credentials scoped to the job, not the run

The agent gets read access to a whole system because one task needs it. Every run inherits everything, whether or not this run's task does.

03
Redaction is priced out

De-identification pipelines cost engineering quarters, so teams ship without them and rely on the prompt asking the model to be careful. Nothing checks that it was.

04
Telemetry was never centralised

Approved agents log to five different places and unapproved ones log nowhere. Data exposure across the fleet is not a number anyone can produce.

§03 / HOW YOU CATCH ITloop: observe → evaluate
How you catch it

How the record bounds it

Prefactor records what every run accessed and evaluates it against the agent's job, so a repeat outside scope becomes a caught event, not a discovery.

Connect

Every agent into one record. Native SDKs for common frameworks, a TypeScript and Python core SDK for anything custom, and OpenTelemetry ingest for closed tools. Data reads and tool calls land as they happen.

Observe

What it accessed, and what it did with it. Each run records the data the agent read, the tools it called, and the output it produced, tagged to that run, queryable per agent.

Evaluate

A scope check on every run. Each run is checked against the agent's job: did it touch data outside its task, did restricted fields reach an output or a tool call. A repeat outside scope is a verdict, not a lucky catch.

Surface

Shadow agents show up. With telemetry centralised, anything emitting runs appears in the fleet view, so the unregistered agent moving customer data stops being unknowable.

§04 / HOW YOU FIX ITloop: act → improve
How you fix it

Hold it, narrow it, prove it

Catching a repeat is the start. The loop is what turns exposure into a bounded, answerable record.

Act

A breach is held before output leaves. A run that puts restricted data in an output or a tool call can be held or escalated to a person before the action completes, so the repeat stays inside the record instead of reaching a customer.

Improve

Narrow scope where it actually leaked. The record shows which prompt, tool, or credential let restricted data in, so the fix is a narrower grant or a redacted field, landed where the leak happened and verified on the next runs.

Prove

"What did it touch?" becomes a lookup. SOC 2, ISO 27001, and sector rules ask what data your agents accessed. The per-run record answers in minutes, with an artefact you can hand to an auditor or your own security team.

A support agent scoped to tickets was found reading billing records to answer refund questions. The scope check flagged the reads, held one output that contained card details before it sent, and escalated to the owning team. The fix was a narrower credential, and the following week's runs proved it held. Illustrative, but this is the standard shape of the catch.

§06 / QUESTIONSfaq: the common ones
Questions
How do I know what data my AI agents accessed?
Record every run: the data read, the tools called, the output produced, tagged to the agent and the run. With that record, "what did it touch" is a query per agent or per dataset, not a reconstruction from scattered logs.
How do I stop an AI agent from leaking PII?
You cannot bound a non-deterministic agent by reviewing its prompt. Check every run for data outside the agent's scope, and give the check the authority to hold or escalate an output before it leaves. Prevention is the scope; the run-time check is what enforces it.
How do I find shadow AI and unregistered agents?
Centralise agent telemetry into one record. Anything emitting runs appears in the fleet view, so agents nobody registered surface next to the approved ones, and unapproved tool usage stops being invisible by construction.
What do SOC 2 and ISO 27001 require for AI agents?
Audit regimes require you to show what data your systems, including agents, accessed and how access was controlled. A per-run record of reads, tool calls, and outputs is the artefact that answers those questions as a lookup rather than a project.
Where does Prefactor itself store the run record?
Prefactor's primary infrastructure runs in Australia; for enterprise engagements it deploys where your data needs to live, in your region or your environment.

See it in action on a fleet like yours

Book a demo and we will put one of your agents in the record: every run watched for scope, breaches held before output leaves, and the audit question answered as a lookup.

Agent Performance Platform
Unified performance platform for agents, authentication, and risk management
All Systems Operational
3Global Agents
7Instances
5Services
12%Human Intervene
4High Risk
$2,360Monthly Spend
Mission ControlLive agent health with 7-day activity heartbeat
Claims Proc...68
$330/moRed
Claims Proc...65
$160/moRed
Claims Proc...82
$170/moAmber
ChatGPT74
$150/moAmber

See how every agent performs, and make it better

Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.