Auditors are pragmatic — they want evidence that controls operate as designed, on a continuous basis. This page covers what NIST AI RMF reviewers expect for AI agents and how to produce it.
What auditors actually look for
- Continuous, dated evidence — not point-in-time. Logs that show controls were operating every day.
- Override and intervention records — proof humans retained control where required.
- Eval results tied to agent versions — proof of quality testing on the version that ran.
- Risk decisions tied to changes — every policy change, prompt edit, version promotion shows who decided, what risk was assessed, what was approved.
- Incident records — even minor incidents documented. Empty incident logs raise suspicion.
- Plain-language documentation — readable by non-technical reviewers.
Evidence packages
Agent inventory + ownership Exportable CSV with agent ID, owner, environment, last modified, last reviewed.
Access reviews Quarterly export of agent capabilities and tool access scopes.
Change records Every policy, prompt, and version change with who/what/when/why.
Monitoring evidence Sample of monitoring dashboards, alert history, response records.
Tamper-evident logs Cryptographic hash chain documentation; audit log access records.
Incident records Full incident timeline with detection, triage, response, post-mortem.
Eval and quality records Eval suite definitions, datasets, scores over time.
How Prefactor produces this evidence
- Every agent action is logged tamper-evidently at runtime
- Every policy/prompt/version change is recorded with full lineage
- Eval results are tied to agent versions and dated
- Incident workflow captures full timeline
- Auditor-ready exports available per agent, per quarter, per framework
Common evidence gaps
1. Logs exist but aren't tamper-evident.
2. No record of overrides — system allows it but nobody uses it.
3. Eval ran once at launch; not tied to current production version.
4. Sub-processor changes happened but aren't documented.
5. Incident response runbooks predate AI agents.
Related
Get a readiness review
[Book a briefing →]