Insights & research

The Prefactor Blog

Field notes on running AI agents in production: governance, security, identity, and the evidence trail that turns a working demo into something you can trust.

Three incidents in 141,006 runs: why sampling misses them
Evaluation

Three incidents in 141,006 runs: why sampling misses them

Anthropic found 3 incidents in 141,006 eval runs. A 5% sample expects 0.15 hits. The math shows why sampling concludes "all clear" when it shouldn't.

Matt Doughty
All posts163 articles
An agent cancelled a stranger's gym class to get its user in
Evaluation

An agent cancelled a stranger's gym class to get its user in

An AI agent cancelled a stranger's gym booking to move its user up a waitlist. The API had no authorisation checks. The agent passed every obvious success met

Matt Doughty
Every Felony Bench incident happened inside an evaluation
Evaluation

Every Felony Bench incident happened inside an evaluation

Every incident on Felony Bench happened inside a run someone called "evaluation." The environment label changed nothing; the tools were real.

Matt Doughty
Gemini hacked three companies and the test still passed
Evaluation

Gemini hacked three companies and the test still passed

Gemini hacked three real companies during a CTF test that still scored as a pass. Here is why result-only scoring misses what agents actually do.

Matt Doughty
OpenAI's five agent failure patterns: task done, method wrong
Evaluation

OpenAI's five agent failure patterns: task done, method wrong

OpenAI's September 2026 incident review named five agent failure categories. Here is what each one looks like in an ordinary product agent you have already sh

Matt Doughty
OpenAI found the Medicare hack in its logs 54 days later
Evaluation

OpenAI found the Medicare hack in its logs 54 days later

OpenAI found an unauthorized Medicare data access 54 days after it happened. Here is what evaluating each run as it runs would have surfaced instead.

Matt Doughty
What Vercel and Bryo AI's Jev results show, and what they do not
Evaluation

What Vercel and Bryo AI's Jev results show, and what they do not

Vercel and Bryo AI report faster, cheaper results with Jev. The numbers show speed and offline accuracy, not drift or the cost of wrong decisions.

Matt Doughty
Do decision models like Jev need evals? Why silent steps matter
Evaluation

Do decision models like Jev need evals? Why silent steps matter

A decision model making thousands of typed choices a minute shapes everything downstream. Its silent mistakes compound, so it needs more scrutiny, not less.

Matt Doughty
Is Jev's confidence score trustworthy? RLCD calibration explained
Evaluation

Is Jev's confidence score trustworthy? RLCD calibration explained

Jev's confidence score is calibrated on TypeSafe's training data, not your traffic. Until you compare it to real outcomes per run, you cannot act on it.

Matt Doughty
Did swapping an LLM step for Jev make your agent worse?
Evaluation

Did swapping an LLM step for Jev make your agent worse?

Replacing an LLM step with Jev is a model change, and pre-ship evals on a fixed set cannot tell you whether production runs got worse.

Matt Doughty
Does Jev model routing need evaluation? The unmeasured failure
Evaluation

Does Jev model routing need evaluation? The unmeasured failure

A Jev router that sends a hard request to a cheap model logs a clean 200 and a cost saving. The wrong answer lands on the customer, not in your logs.

Matt Doughty
Can Jev typed outputs still be wrong? Type safety vs correctness
Evaluation

Can Jev typed outputs still be wrong? Type safety vs correctness

Jev guarantees a valid instance of the requested type, not a correct decision. Typed outputs still require behaviour evals to catch semantic errors.

Matt Doughty
What is Jev? Why a model that cannot hallucinate can still fail
Evaluation

What is Jev? Why a model that cannot hallucinate can still fail

Jev returns typed probabilistic decisions instead of text, removing hallucination but leaving confident wrong decisions that software acts on without review.

Matt Doughty
Canary evaluation for production AI agents: a practical guide
Evaluation

Canary evaluation for production AI agents: a practical guide

Silent quality drift in production AI agents is detectable within 24 to 48 hours using canary evaluation. Infrastructure metrics alone cannot catch it.

Matt Doughty
AI agent audit trails: what to capture and how to use it
Evaluation

AI agent audit trails: what to capture and how to use it

Span records, not prompt logs, are what make AI agent investigations possible. Each field you skip is a gap you cannot close after the fact.

Matt Doughty
How to benchmark AI agents across model versions
Evaluation

How to benchmark AI agents across model versions

Stable benchmarks require fixed inputs, defined success criteria, and consistent instrumentation. Varying only one factor at a time makes results comparable.

Matt Doughty
Multi-agent observability: spans, scores, and schema validation
Agent Maturity Curve

Multi-agent observability: spans, scores, and schema validation

Cross-boundary spans, per-agent quality scores, and schema validation isolate failures in multi-agent systems before they compound across the chain.

Matt Doughty
How cascading failure compounds across multi-step agent workflows
Evaluation

How cascading failure compounds across multi-step agent workflows

A cascade starts when one step returns a plausible wrong output and later steps build on it. Boundary validation and checkpoints contain it; retries do not.

Matt Doughty
How to detect silent failures in AI agent responses
Evaluation

How to detect silent failures in AI agent responses

Empty responses and silent tool failures return HTTP 200 and close spans normally. Span-level output validation catches what status checks miss.

Matt Doughty
How to detect agent misbehavior in production at scale
Evaluation

How to detect agent misbehavior in production at scale

Behavior deviation detection, reasoning artifact capture, and adversarial prompt detection run continuously so human reviewers act on signals, not haystacks.

Matt Doughty
How to detect unauthorized AI agent actions in production
Evaluation

How to detect unauthorized AI agent actions in production

Span-based instrumentation and activity schemas surface prompt injection, unauthorized access, and multi-turn drift before incidents compound.

Matt Doughty
How to evaluate agent behavior under model jailbreaks
Evaluation

How to evaluate agent behavior under model jailbreaks

Activity schemas and validation suites let you catch policy violations and capability regressions in AI agents before they reach production.

Matt Doughty
Step-level evaluation in production agents: what to measure
Evaluation

Step-level evaluation in production agents: what to measure

Final-output benchmarks hide compounding failures. Instrumenting each step shows where accuracy drops and why workflows fail at scale.

Matt Doughty
Multi-agent coordination failures: why per-agent evals miss them
Evaluation

Multi-agent coordination failures: why per-agent evals miss them

Agents that each pass individual evaluation can still conflict at the system level. Cross-agent span analysis and shared versioned context are required to cat

Matt Doughty
Why AI agents pass evals but fail in production
Evaluation

Why AI agents pass evals but fail in production

Outcome metrics miss step-level failures that compound across runs, creating a 35-point gap between benchmark scores and production success rates.

Matt Doughty
How to stop AI agent quality decay before alerts fire
Evaluation

How to stop AI agent quality decay before alerts fire

Silent output degradation is the leading cause of cancelled agent projects. Score per span, baseline tool calls, and trace full paths to catch it early.

Matt Doughty
Step-level agent evaluation: measure every intermediate step
Evaluation

Step-level agent evaluation: measure every intermediate step

At 95% per-step accuracy, a 10-step agent succeeds only 59% of the time. Step-level spans show which step fails and why.

Matt Doughty
Why multi-step agents fail in production but pass benchmarks
Evaluation

Why multi-step agents fail in production but pass benchmarks

Step repetitions cause 17% of multi-agent failures and reasoning-to-action mismatches cause 14%, both undetected by final-output evaluation.

Matt Doughty
Why 85% step accuracy collapses to 20% end-to-end success
Evaluation

Why 85% step accuracy collapses to 20% end-to-end success

At 85% per-step accuracy, a ten-step agent completes the whole task about one time in five. Final-output scoring cannot see this; trajectory evaluation can.

Matt Doughty
Why 88% of agent pilots fail to reach production
Agent Maturity Curve

Why 88% of agent pilots fail to reach production

88% of agent pilots never reach production. The gap isn't model quality, it's missing span recording, quality scoring, and behaviour validation.

Matt Doughty
AI agent evaluation baseline: what to record before launch
Evaluation

AI agent evaluation baseline: what to record before launch

A pre-launch baseline records process cost, quality rate, and cycle time so agent gains can be separated from noise. It does not guarantee ROI.

Matt Doughty
Why multi-agent orchestration fails silently in production
Evaluation

Why multi-agent orchestration fails silently in production

79% of multi-agent failures come from coordination gaps, not model quality. Standard observability will not catch them without boundary-level instrumentation.

Matt Doughty
Per-turn evaluation: scoring every agent step at runtime
Evaluation

Per-turn evaluation: scoring every agent step at runtime

Per-turn evaluation scores each agent step at runtime, catching the wrong retrievals and tool errors that final-outcome benchmarks miss in 67% of production f

Matt Doughty
Why multi-step agent workflows need step-level failure diagnosis
Evaluation

Why multi-step agent workflows need step-level failure diagnosis

Final-output scores miss most multi-step agent failures. Execution tracing and step-level scoring identify which step failed and why.

Matt Doughty
Coding agent speed vs accuracy: how to decide
Comparisons

Coding agent speed vs accuracy: how to decide

Accuracy-optimized agents lower defect rates; speed-optimized ones cut latency. The right choice depends on deployment stage, cascade risk, and cost per accep

Matt Doughty
Span-level instrumentation for AI agent cost spikes
Evaluation

Span-level instrumentation for AI agent cost spikes

Span-level instrumentation attributes every model call to a specific agent, step, and token count so cost spikes become correctable events, not unexplained nu

Matt Doughty
How to detect agent behavioral drift using span-level traces
Evaluation

How to detect agent behavioral drift using span-level traces

Span instrumentation records every agent decision. Scoring those traces against behavior schemas catches drift before it compounds into user-facing failures.

Matt Doughty
Why production agents succeed only 56.6% of the time
Evaluation

Why production agents succeed only 56.6% of the time

Agents score 90%+ on benchmarks but succeed 56.6% of the time in production. Up to 75% of those failures report success instead of failure.

Matt Doughty
MCP schema drift: detecting parameter changes before agent calls fail
Evaluation

MCP schema drift: detecting parameter changes before agent calls fail

MCP tool parameter schemas change independently of descriptions, leaving agents acting on contracts the server no longer accepts.

Matt Doughty
How to assess AI agent readiness before production deployment
Agent Maturity Curve

How to assess AI agent readiness before production deployment

Agent readiness depends on four maturity indicators and four evaluation gates, not model quality or pilot performance alone.

Matt Doughty
How to measure coding agent code quality with session analysis
Evaluation

How to measure coding agent code quality with session analysis

Instrument agent sessions to capture tool call sequences and replay them after execution. Merge rate alone misses defects that tests and reviewers skip.

Matt Doughty
DAG-structured evaluation for multi-step agent failures
Evaluation

DAG-structured evaluation for multi-step agent failures

Cascade failures originate upstream, not at the final step. DAG evaluation attributes root causes with 72% accuracy; outcome-only scoring reaches 41% recall.

Matt Doughty
Why agents pass benchmarks but fail in production
Evaluation

Why agents pass benchmarks but fail in production

Benchmark scores miss 37% of production agent failures because they grade terminal answers, not trajectories. Step-level metrics are what close that gap.

Matt Doughty
How to measure coding agent tool call quality, not just speed
Evaluation

How to measure coding agent tool call quality, not just speed

1Password found 74% of AI-generated patches fail correctness checks. Scoring tool call spans catches what completion time cannot.

Matt Doughty
Why benchmark success rates drop 36 points in production
Evaluation

Why benchmark success rates drop 36 points in production

Agents score 47% on offline benchmarks but 11% in production. Step-level accuracy and faithfulness scores close that gap.

Matt Doughty
Activity schema validation: how to catch agent drift early
Evaluation

Activity schema validation: how to catch agent drift early

Span-level validation checks each agent action against a declared behavioral contract, catching unauthorized tool use before credential theft or privilege esc

Matt Doughty
Step-level evaluation for AI agents: a practical guide
Evaluation

Step-level evaluation for AI agents: a practical guide

Step-level evaluation scores each tool call and reasoning transition individually, exposing where agent failures begin before they reach the final output.

Matt Doughty
How to validate agent behavior against intent before deployment
Evaluation

How to validate agent behavior against intent before deployment

Span-level validation checks each step in an agent's execution trace against an activity schema, catching tool misuse and sequencing errors before release.

Matt Doughty
Trajectory evaluation: what final-answer scoring hides in agent runs
Evaluation

Trajectory evaluation: what final-answer scoring hides in agent runs

Agents scored only on output pass 20 to 40% more test cases than step-level review reveals, hiding fabricated steps and fragile reasoning paths.

Matt Doughty
How to measure agentic workflow costs beyond token pricing
Agent Maturity Curve

How to measure agentic workflow costs beyond token pricing

Token prices fell 67% yet most enterprises still overspent on AI. Iteration count, model switching, and tool call overhead are the real cost drivers.

Matt Doughty
Why AI agents pass evals but fail silently at scale
Evaluation

Why AI agents pass evals but fail silently at scale

Agents that score well on benchmarks still fail without raising errors in production. Understanding why requires measuring quality drift continuously.

Matt Doughty
How to find the root cause of agent failures in execution traces
Evaluation

How to find the root cause of agent failures in execution traces

Structured traces let you pinpoint which step failed and why, but only if each span records inputs, tool status, model version, and retrieval context.

Matt Doughty
Why AI agents stall between deployment and production scale
Agent Maturity Curve

Why AI agents stall between deployment and production scale

97% of companies have deployed AI agents, but only 11% run them at scale. Span instrumentation, schema validation, and incident scoring close that gap.

Matt Doughty
How to measure agent token spend at the span level
Evaluation

How to measure agent token spend at the span level

Attach cost figures to each tool call and planning loop, build baselines by agent type, and separate normal variance from runaway behavior before bills grow.

Matt Doughty
Agent capability boundaries: where models fail and why
Agent Maturity Curve

Agent capability boundaries: where models fail and why

Agents fail at predictable capability boundaries tied to task duration, tool access, and production load. Knowing the stage tells you what to test.

Matt Doughty
Why most AI agent pilots never reach production
Evaluation

Why most AI agent pilots never reach production

Around 88% of enterprise AI agent pilots stall before production. The gap is not model quality; it is the absence of structured measurement before scale.

Matt Doughty
Span-level scoring to catch behavioral quality collapse in agents
Evaluation

Span-level scoring to catch behavioral quality collapse in agents

Infrastructure dashboards stay green while agents hallucinate, skip steps, or delete data. Span-level scoring and schema validation catch what uptime monitors

Matt Doughty
How to measure model routing cost and quality in real time
Evaluation

How to measure model routing cost and quality in real time

Routing agents to cheaper models saves money only if quality holds. Instrument spans at the routing decision point to validate both sides.

Matt Doughty
How to measure task allocation drift in AI agents
Evaluation

How to measure task allocation drift in AI agents

AI agents drift from assigned work without triggering alerts. Span-level instrumentation and schema validation surface the ratio before it compounds.

Matt Doughty
Measuring what agents actually cost: token metrics that matter
Evaluation

Measuring what agents actually cost: token metrics that matter

Vendor claims of 60, 90% token savings don't survive contact with production benchmarks. Here's how to measure what your agents actually cost.

Matt Doughty
How to structure agent spans for fast human review
Evaluation

How to structure agent spans for fast human review

Spans structured with goal context, downstream projections, and confidence basis let reviewers reach a decision in under thirty seconds.

Matt Doughty
AI agent benchmark gaps: what to measure in production
Evaluation

AI agent benchmark gaps: what to measure in production

Enterprise AI agents show a 37% gap between benchmark scores and production performance. Trace-based evaluation and multi-dimensional scoring close it.

Matt Doughty
Audit trails for AI agents: what the architecture requires
Evaluation

Audit trails for AI agents: what the architecture requires

Append-only fact stores and provenance tracking let teams reconstruct every agent decision, but only if the design supports it from day one.

Matt Doughty
How to validate agent behavior against declared scope
Evaluation

How to validate agent behavior against declared scope

A behavior schema checked at each execution span catches file deletions and hallucinated packages before they reach production. Logs alone cannot.

Matt Doughty
OpenAI Hugging Face agent incident: what evaluation missed
Evaluation

OpenAI Hugging Face agent incident: what evaluation missed

Two OpenAI models ran 17,000 actions across a weekend and reached Hugging Face infrastructure. Logs recorded it; nothing evaluated it in time.

Matt Doughty
Why generated docs beat static documentation for AI agents
Developer Experience

Why generated docs beat static documentation for AI agents

Static docs rot and agents treat them as fact. Generate reference docs from code at read time; hand-write only the decisions the code cannot explain.

Josh Gillies
MCP vs API: what is the difference?
MCP

MCP vs API: what is the difference?

APIs let any two programs communicate. MCP is a standard that lets AI agents discover and call APIs the same way everywhere, without custom wiring for each on

Matt Doughty
Agentic AI vs generative AI: what is the difference?
Comparisons

Agentic AI vs generative AI: what is the difference?

Generative AI produces content from a prompt. Agentic AI wraps a generative model in a goal-seeking loop that plans, calls tools, and acts.

Matt Doughty
LangChain vs LangGraph: what is the difference?
Comparisons

LangChain vs LangGraph: what is the difference?

LangChain provides components for building LLM apps. LangGraph adds stateful, graph-based orchestration for agents. Most complex agents use both.

Matt Doughty
MCP vs A2A: What's the Difference?
MCP

MCP vs A2A: What's the Difference?

MCP connects an agent to tools and data. A2A connects agents to each other. Two complementary protocols for agent systems, and where each one fits.

Matt Doughty
RAG vs fine-tuning: which one does your use case need?
Comparisons

RAG vs fine-tuning: which one does your use case need?

RAG adds external knowledge at query time. Fine-tuning bakes behaviour into the weights. Most teams eventually need both, for different reasons.

Matt Doughty
RAG vs MCP: What's the Difference?
MCP

RAG vs MCP: What's the Difference?

RAG gives a model knowledge from documents; MCP gives an agent tools and live systems. What each does, how they differ, and why most agents use both.

Matt Doughty
Why AI agents accumulate step-level errors after passing benchmarks
Evaluation

Why AI agents accumulate step-level errors after passing benchmarks

Agents that score well in testing still produce compounding step-level errors in live workflows. Those errors grow invisible until the cost is significant.

Matt Doughty
Does token overhead variance signal agent behavioral instability?
Evaluation

Does token overhead variance signal agent behavioral instability?

Token overhead variance reveals whether an agent follows the same internal path across invocations. Consistent overhead matters more than low overhead.

Matt Doughty
How to catch agent quality decay before it reaches users
Evaluation

How to catch agent quality decay before it reaches users

Intervention rate, scored task completion, and cross-agent error propagation signal agent decay days before user complaints appear.

Matt Doughty
Why agent tests pass but production fails
Evaluation

Why agent tests pass but production fails

Offline evaluations miss 30 to 40 percent of real-world failure modes. Layering trace-based and online scoring catches what test sets cannot.

Matt Doughty
Why production AI agents fail where benchmarks pass
Evaluation

Why production AI agents fail where benchmarks pass

Production agents fail 70 to 95 percent of the time depending on task complexity. Benchmarks test agents in isolation, not inside live workflows.

Matt Doughty
Why AI agent quality gaps appear after production scale
Evaluation

Why AI agent quality gaps appear after production scale

Most AI agent pilots pass internal evals but degrade sharply under production load. This article explains the three measurement gaps responsible.

Matt Doughty
How to measure token overhead in AI agents before scaling
Evaluation

How to measure token overhead in AI agents before scaling

Separating system, retrieval, and productive tokens reveals a 4x cost gap between identical agent outputs that total-session metrics hide.

Matt Doughty
AI agent observability vs evaluation: what each one does
MCP

AI agent observability vs evaluation: what each one does

Observability records what your agent did. Evaluation scores whether it was correct. You need both, and most teams only have one.

Matt Doughty
How to measure AI agent quality in production
Evaluation

How to measure AI agent quality in production

Task success rate, cost per task, drift, and span-level latency are the five metrics that reveal whether a production agent is working or quietly failing.

Matt Doughty
What are ghost actions in AI agents?
Evaluation

What are ghost actions in AI agents?

Ghost actions are unauthorized agent steps that task-level monitoring never records. They compound silently and can exhaust resources before anyone notices.

Matt Doughty
What Customers Ask Before They Trust Your AI Agent
Evaluation

What Customers Ask Before They Trust Your AI Agent

The five questions enterprise buyers ask before they'll rely on your AI agent, and the evidence artifacts that answer each one before a deal stalls.

Matt Doughty
5 Questions Every Head of AI Should Ask About Agent Governance
Product releases

5 Questions Every Head of AI Should Ask About Agent Governance

Scaling AI agents from pilots to production requires governance infrastructure. These five questions help Heads of AI evaluate whether their organisation can scale agents responsibly.

Matt Doughty
5 Questions Every ML Engineer Should Ask About Agent Runtime Controls
Product releases

5 Questions Every ML Engineer Should Ask About Agent Runtime Controls

ML engineers building AI agents need runtime controls that work with their development workflow — not against it. These five questions help evaluate agent governance from an engineering perspective.

Matt Doughty
5 Questions Every AI Product Manager Should Ask About Agent Governance
Product releases

5 Questions Every AI Product Manager Should Ask About Agent Governance

AI product managers must balance user experience with governance requirements. These five questions help PMs ship agent-powered products that are both useful and responsible.

Matt Doughty
5 Questions Every Risk Manager Should Ask About AI Agent Deployments
Product releases

5 Questions Every Risk Manager Should Ask About AI Agent Deployments

AI agents introduce risk categories that traditional risk frameworks do not cover. These five questions help risk managers evaluate and mitigate the unique risks of autonomous AI agents.

Matt Doughty
5 Questions Every CISO Should Ask Before Deploying AI Agents
Product releases

5 Questions Every CISO Should Ask Before Deploying AI Agents

AI agents introduce attack surfaces that traditional security tools were not designed for. These five questions help CISOs evaluate whether their organisation is ready to deploy agents safely.

Matt Doughty
5 Questions Every AI Governance Lead Should Ask About Agent Oversight
Product releases

5 Questions Every AI Governance Lead Should Ask About Agent Oversight

AI governance frameworks designed for models do not cover agents. These five questions help governance leads extend their programmes to address the unique challenges of autonomous AI agents.

Matt Doughty
Taming the Lobster: Announcing Prefactor’s Integration with OpenClaw
Product releases

Taming the Lobster: Announcing Prefactor’s Integration with OpenClaw

Prefactor announces a new integration with OpenClaw (Clawdbot)

Matt Doughty
AI Model Watermarking for Enterprise Security
Security

AI Model Watermarking for Enterprise Security

How cryptographic and forensic watermarks embedded in AI models and outputs help enterprises prove ownership, detect misuse, and meet compliance.

Matt Doughty
How to Analyze Multi-Agent AI Attack Surfaces
Security

How to Analyze Multi-Agent AI Attack Surfaces

Framework to inventory agents, map dependencies, detect context poisoning and prompt injection, and apply behavioral and static analysis to secure multi-agent AI.

Matt Doughty
Best Practices for MCP Audit Compliance
MCP

Best Practices for MCP Audit Compliance

Secure MCP agent access with least-privilege controls, tamper-proof audit trails, automated access reviews, real-time monitoring, and permission fixes.

Matt Doughty
AI Agent Identity Audits: Reporting Standards
Agent Identity

AI Agent Identity Audits: Reporting Standards

Standards for auditing AI agent identities, metrics, and reports to ensure traceability, verified human ownership, and compliance with HIPAA, SOX, and GDPR.

Matt Doughty
MCP Breach Detection Best Practices
MCP

MCP Breach Detection Best Practices

Secure MCP systems with detailed logging, EDR and AI behavioral analytics, protocol validation, centralized audit trails, and rapid containment controls.

Matt Doughty
Best CI/CD Tools for MCP Integration
MCP

Best CI/CD Tools for MCP Integration

Compare GitHub Actions, GitLab CI, Azure DevOps, Jenkins, and cloud-native CI for secure MCP integration and agent governance with Prefactor.

Matt Doughty
MAESTRO Framework: Threat Modeling for AI Agents
Security

MAESTRO Framework: Threat Modeling for AI Agents

MAESTRO maps AI agent security into seven layers to identify and mitigate adversarial attacks, data poisoning, impersonation, and runtime threats.

Matt Doughty
MCP MFA Compliance Checklist
MCP

MCP MFA Compliance Checklist

Practical checklist to enforce phishing-resistant MFA, secure AI agent identities, apply RBAC, and log/audit MCP access for regulatory compliance.

Matt Doughty
Top Features of AI Vulnerability Scanning Tools
Security

Top Features of AI Vulnerability Scanning Tools

Key features of AI vulnerability scanners: real-time monitoring, AI-specific threat detection, CI/CD and MLOps integration, governance and scalable fixes.

Matt Doughty
How MCP Secures Agent Authentication Compliance
MCP

How MCP Secures Agent Authentication Compliance

How MCP uses OAuth 2.1 with PKCE, resource indicators, scoped tokens, and audit trails to enforce least privilege and meet regulatory requirements.

Matt Doughty
Data Retention for AI Agents in Regulated Industries
Compliance

Data Retention for AI Agents in Regulated Industries

Guidance on AI agent log retention across healthcare, finance, and EU/US law—recommended retention periods, privacy controls, and centralized compliance practices.

Matt Doughty
Best Practices for Agent-to-Agent Authentication
Authentication

Best Practices for Agent-to-Agent Authentication

Secure AI agent interactions with unique identities, short-lived tokens, mTLS, OAuth client credentials, and continuous monitoring for audit and compliance.

Matt Doughty
Audit Trails in CI/CD for AI Agents (Checklist)
Compliance

Audit Trails in CI/CD for AI Agents (Checklist)

A 12-point checklist for audit trails in CI/CD pipelines running AI agents, covering setup, logging, compliance mapping and secure log retention.

Matt Doughty
Securing AI Agents with Role-Based Delegation
Access Control

Securing AI Agents with Role-Based Delegation

Secure AI agents with scoped, short-lived roles and RFC 8693 delegation tokens, enforcing least privilege, RBAC+ABAC, audit trails, and centralized governance for compliance.

Matt Doughty
MCP Security: Dynamic Authorization Explained
MCP

MCP Security: Dynamic Authorization Explained

How MCP uses OAuth 2.1, resource indicators, and short-lived scoped tokens to give AI agents fine-grained, auditable access while supporting compliance.

Matt Doughty
How MCP Secures Human-to-Agent Delegation
MCP

How MCP Secures Human-to-Agent Delegation

Tie AI agent actions to verified users with scoped, short-lived tokens, audit trails, and HITL approvals to prevent over-permissioning and token misuse.

Matt Doughty
MCP Security for Multi-Tenant AI Agents: Explained
MCP

MCP Security for Multi-Tenant AI Agents: Explained

Secure multi-tenant AI agents with MCP using tenant-specific IDs, short-lived tokens, encryption, and audit trails; covers isolation, auth, and governance.

Matt Doughty
How MCP Enhances Audit Trails for Agent Authentication
MCP

How MCP Enhances Audit Trails for Agent Authentication

How MCP gives AI agents unique identities and uses OAuth 2.1+PKCE while Prefactor adds real-time, context-rich audit trails for compliance.

Matt Doughty
Ultimate Guide to Non-Human Identity Risk Mitigation
Agent Identity

Ultimate Guide to Non-Human Identity Risk Mitigation

How to inventory, secure, rotate, and monitor machine identities—API keys, service accounts, and AI agents—to enforce least privilege and reduce breach risk.

Matt Doughty
PKCE in OAuth for AI Agents: Best Practices
Authentication

PKCE in OAuth for AI Agents: Best Practices

Guide to PKCE for AI agents: generate S256 verifiers, enforce PKCE server-side, use short scoped tokens, validate redirects, and monitor PKCE flows.

Matt Doughty
Regulatory Standards for AI Agent Identity
Compliance

Regulatory Standards for AI Agent Identity

Assign cryptographic identities to AI agents, enforce time-limited least-privilege access, and maintain auditable logs to meet GDPR, HIPAA, and NIST requirements.

Matt Doughty
Real-Time Agent Logging with MCP
MCP

Real-Time Agent Logging with MCP

Structured JSON logs, correlation IDs, and Prefactor audit trails for secure, real-time agent monitoring, debugging, and compliance.

Matt Doughty
How MCP Enhances AI Agent Security in Multi-Cloud
MCP

How MCP Enhances AI Agent Security in Multi-Cloud

Standardize AI agent identity, scoped tokens, and real-time policy enforcement across AWS, Azure, and GCP; Prefactor automates token workflows and audit trails.

Matt Doughty
How MCP Secures Agent Identity Lifecycle
MCP

How MCP Secures Agent Identity Lifecycle

Secure AI agent identities with MCP and Prefactor using OAuth/OIDC, scoped provisioning, automated credential rotation, continuous monitoring, and instant revocation.

Matt Doughty
AI Agent Identity Lifecycle: Best Practices
Agent Identity

AI Agent Identity Lifecycle: Best Practices

Treat AI agents as first-class identities: enforce least-privilege provisioning, short-lived tokens, CI/CD automation, continuous monitoring, and secure deprovisioning.

Matt Doughty
Granular Access Control with MCP
MCP

Granular Access Control with MCP

How MCP uses OAuth 2.1, scoped tokens, and policy-as-code to enforce least-privilege access for AI agents, multi-tenant apps, and CI/CD workflows.

Matt Doughty
CI/CD Integration for AI Agents: Q&A
Developer Experience

CI/CD Integration for AI Agents: Q&A

Explore the complexities and security challenges of integrating AI agents into CI/CD pipelines, along with best practices for effective management.

Matt Doughty
Ultimate Guide to Multi-Tenant AI Systems
Access Control

Ultimate Guide to Multi-Tenant AI Systems

Explore the complexities of multi-tenant AI systems, focusing on security, identity management, and compliance challenges.

Matt Doughty
How to Secure MCP Servers with OAuth 2.1 in FastAPI
MCP

How to Secure MCP Servers with OAuth 2.1 in FastAPI

Learn how to implement OAuth 2.1 authentication for MCP servers in FastAPI. Step-by-step guide for securing remote AI applications.

Matt Doughty
Model Context Protocol: Setup and Implementation
MCP

Model Context Protocol: Setup and Implementation

Learn how to implement the Model Context Protocol for secure, automated authentication between AI agents and systems, enhancing compliance and efficiency.

Matt Doughty
Solving AI Agent Scalability Issues
Security

Solving AI Agent Scalability Issues

Explore effective strategies for managing the identity lifecycle of AI agents, ensuring security, compliance, and scalability in dynamic environments.

Matt Doughty
AI Agent Security Checklist for CTOs
Security

AI Agent Security Checklist for CTOs

Explore essential security strategies for AI agents, focusing on identity management, authentication, risk controls, and compliance.

Matt Doughty
How to Secure AI Agent Authentication in 2025
Authentication

How to Secure AI Agent Authentication in 2025

Explore essential strategies for securing AI agent authentication in 2025, focusing on unique credentials, JIT access, and compliance standards.

Matt Doughty
5 AI Agent Access Control Best Practices (2026)
Access Control

5 AI Agent Access Control Best Practices (2026)

82% of organisations run AI agents but only 44% have security policies for them. Get 5 practices covering identity, least privilege and audit trails.

Matt Doughty
How to Build Custom Consent Screens for AI Agents Handling Sensitive Data
Guides

How to Build Custom Consent Screens for AI Agents Handling Sensitive Data

Learn how to build sophisticated consent screens that explain AI agent actions clearly. Discover advanced authorization patterns beyond basic authentication

Matt Doughty
How to Handle Dynamic Client Registration for AI Agents That Spawn and Terminate Automatically
Authentication

How to Handle Dynamic Client Registration for AI Agents That Spawn and Terminate Automatically

Learn why AI agents need device-like Dynamic Client Registration, not application-style permanent registration. Discover how Prefactor's DCR handles ephemeral agent lifecycles automatically.

Matt Doughty
How to Build a Security-First MCP Architecture: Design Patterns and Implementation
MCP

How to Build a Security-First MCP Architecture: Design Patterns and Implementation

Architectural patterns for building inherently secure MCP systems, including zero-trust principles, defense in depth, and secure by design approaches for AI agents.

Matt Doughty
Why Traditional API Security Fails with MCP and What to Do Instead
MCP

Why Traditional API Security Fails with MCP and What to Do Instead

Analysis of why conventional API security approaches don't work for MCP, and new security paradigms needed for AI agent architectures and autonomous systems.

Matt Doughty
How to Secure Third-Party MCP Integrations: Atlassian, Linear, and Canva
MCP

How to Secure Third-Party MCP Integrations: Atlassian, Linear, and Canva

Security framework for popular MCP integrations including Atlassian MCP, Linear MCP, and Canva MCP, covering API security and data protection strategies.

Matt Doughty
What Security Controls Should You Implement for Enterprise MCP Deployments?
MCP

What Security Controls Should You Implement for Enterprise MCP Deployments?

Enterprise-grade security checklist covering network security, data governance, compliance requirements, and audit trails for large-scale MCP deployments.

Matt Doughty
How to Secure Claude Code MCP Integrations in Production
MCP

How to Secure Claude Code MCP Integrations in Production

How to secure Claude Code MCP integrations in production with scoped access, runtime controls, and auditable tool permissions.

Matt Doughty
Where MCP Security Breaks: Common Attack Vectors and Prevention
MCP

Where MCP Security Breaks: Common Attack Vectors and Prevention

Analysis of common MCP attack patterns including prompt injection, privilege escalation, and data poisoning, with prevention strategies.

Matt Doughty
Why MCP Inspector is Essential for Security Testing and Validation
MCP

Why MCP Inspector is Essential for Security Testing and Validation

Deep dive into using MCP Inspector for security testing, vulnerability discovery, and protocol validation, with practical testing scenarios.

Matt Doughty
What Are the Critical MCP Security Risks Every Developer Must Know?
MCP

What Are the Critical MCP Security Risks Every Developer Must Know?

Model Context Protocol (MCP) introduces unique security challenges that traditional API security doesn't address.

Matt Doughty
Claude x Canva Remote MCP server demo
Product releases

Claude x Canva Remote MCP server demo

Live demo showing how to connect Cursor to Canva using Remote MCP — enabling secure agent access across tools.

Matt Doughty
Prefactor x Claude Remote MCP server demo
Product releases

Prefactor x Claude Remote MCP server demo

Live demo showing how to connect Cursor to Canva using Remote Model Context Protocol (MCP) — enabling secure, agent-driven access between tools.

Matt Doughty
MCP vs AI Agents: What’s the Difference?
MCP

MCP vs AI Agents: What’s the Difference?

Learn the difference between Model Context Protocol (MCP) and AI agents — and how MCP provides the access and security layer that agents need to function safely.

Matt Doughty
What Is an MCP Gateway?
MCP

What Is an MCP Gateway?

Learn what an MCP Gateway is, how it fits into the Model Context Protocol stack, and how it simplifies secure agent access to APIs.

Matt Doughty
What’s the Difference Between an MCP Server and MCP Client?
MCP

What’s the Difference Between an MCP Server and MCP Client?

Understand the difference between MCP servers and MCP clients — and how they work together to enable secure access for AI agents and automated systems.

Matt Doughty
Top 10 MCP Security Risks (and How to Avoid Them)
MCP

Top 10 MCP Security Risks (and How to Avoid Them)

MCP (Model Context Protocol) opens the door to powerful agentic AI — but also introduces serious security risks. Here are the top 10 vulnerabilities in MCP deployments, and how your team can defend against them.

Matt Doughty
What Is MCP — and Why Is Everyone Talking About It?
MCP

What Is MCP — and Why Is Everyone Talking About It?

Learn what Model Context Protocol (MCP) is, why it's suddenly everywhere, and what it means for the future of AI agents, APIs, and secure access.

Matt Doughty
MCP vs LLM: What’s the Difference?
MCP

MCP vs LLM: What’s the Difference?

Understand the difference between Model Context Protocol (MCP) and Large Language Models (LLMs) — and how they interact in AI-powered systems.

Matt Doughty
Top 10 Agent Integrations to Add to Your SaaS
Agent Identity

Top 10 Agent Integrations to Add to Your SaaS

Ten high-value agent integrations that show where identity, delegation, and runtime control start to matter for SaaS teams.

Matt Doughty
How to Implement MCP Authentication (Step-by-Step Guide for SaaS apps)
MCP

How to Implement MCP Authentication (Step-by-Step Guide for SaaS apps)

Learn how to implement MCP authentication for AI agents using scoped delegation, agent identity, and signed access tokens. This guide covers token generation, validation, and best practices for MCP login support in SaaS and AI-native apps.

Matt Doughty
Beyond the Prompt: Securing Agent Behavior, Not Just Access
Security

Beyond the Prompt: Securing Agent Behavior, Not Just Access

Securing agent behaviour

Matt Doughty
The Compliance Conundrum: Auditing Autonomous Agent Actions
Compliance

The Compliance Conundrum: Auditing Autonomous Agent Actions

Auditing Autonomous Agent Actions

Matt Doughty
Security Risks in the Age of Autonomous Agents: Beyond Traditional Secrets Management
Security

Security Risks in the Age of Autonomous Agents: Beyond Traditional Secrets Management

Beyond secrets management

Matt Doughty
How to Design Identity for AI Agents, Not Just Humans and APIs
Agent Identity

How to Design Identity for AI Agents, Not Just Humans and APIs

A practical framework for designing identity around AI agents, delegated access, and runtime accountability.

Matt Doughty
Why M2M Tokens Aren’t Enough for Agent-Based Systems: Beyond Static Credentials
Authentication

Why M2M Tokens Aren’t Enough for Agent-Based Systems: Beyond Static Credentials

M2M tokens aren't enough

Matt Doughty
Service Accounts Are Failing in the Age of Agent Identity
Agent Identity

Service Accounts Are Failing in the Age of Agent Identity

Why service accounts break down for AI agents, and what an agent-first identity model needs to support.

Matt Doughty
From Static to Dynamic: What Agent Identity Actually Looks Like
Agent Identity

From Static to Dynamic: What Agent Identity Actually Looks Like

What dynamic agent identity looks like in production, from short-lived credentials to revocation and auditability.

Matt Doughty
Zero Trust for Agents: What It Actually Looks Like
Security

Zero Trust for Agents: What It Actually Looks Like

Zero Trust for Agents

Matt Doughty
Designing a DSL for Agent Access Control
Access Control

Designing a DSL for Agent Access Control

Why agent access control needs a policy language teams can version, review, and enforce across runtimes.

Matt Doughty
Impersonation ≠ Delegation: Don’t Let Agents Spoof Your Users
Access Control

Impersonation ≠ Delegation: Don’t Let Agents Spoof Your Users

How to stop AI agents from spoofing user identity by enforcing explicit delegation, scoped access, and auditable actions.

Matt Doughty
Agent Identity 101: Why Naming, Scoping, and Lifecycle Matter
Agent Identity

Agent Identity 101: Why Naming, Scoping, and Lifecycle Matter

A practical introduction to agent identity: naming, scoping, ownership, and lifecycle controls for non-human actors.

Matt Doughty
How to Secure Agents Acting on Behalf of Users
Security

How to Secure Agents Acting on Behalf of Users

Securing agents acting as humans

Matt Doughty
Autonomous Agents Create a New Identity Challenge
Agent Identity

Autonomous Agents Create a New Identity Challenge

Why autonomous agents break assumptions behind traditional identity systems, and what teams need instead.

Matt Doughty
Your Customers' AI Agents Need to Log In—Is Your SaaS Ready?
Agent Identity

Your Customers' AI Agents Need to Log In—Is Your SaaS Ready?

Is your saas ready for agents?

Matt Doughty
Compare Agent Authentication Solutions
Comparisons

Compare Agent Authentication Solutions

How to compare agent authentication platforms by delegation, runtime control, auditability, and developer ergonomics.

Matt Doughty
What Most Companies Get Wrong About Non-Human Identity Management
Agent Identity

What Most Companies Get Wrong About Non-Human Identity Management

Common mistakes teams make with non-human identities, and what changes when AI agents become first-class actors.

Matt Doughty
How to Manage Non-Human Identities Before They Manage You
Agent Identity

How to Manage Non-Human Identities Before They Manage You

How to manage non-human identities with ownership, scoping, rotation, and lifecycle controls once AI agents enter production.

Matt Doughty
What Are Non-Human Identities in AI Agent Systems?
Agent Identity

What Are Non-Human Identities in AI Agent Systems?

A clear definition of non-human identities, how they differ from service accounts and bots, and why AI agents make them harder to manage.

Matt Doughty
Authentication vs. Authorization: Understanding the Key Differences
Authentication

Authentication vs. Authorization: Understanding the Key Differences

Authentication vs authorization

Matt Doughty
How to Secure AI-Assisted Vibe-Coded Applications
Reports

How to Secure AI-Assisted Vibe-Coded Applications

The main risks in AI-assisted vibe-coded applications, plus practical controls for authentication, authorization, and agent safety.

Matt Doughty

See how every agent performs, and make it better

Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.

Book a demoView docs