Insights & research

The Prefactor Blog

Field notes on running AI agents in production: governance, security, identity, and the evidence trail that turns a working demo into something you can trust.

Detecting Agent Quality Decay in Production: The Hidden Metric Every Team Misses
Evaluation

Detecting Agent Quality Decay in Production: The Hidden Metric Every Team Misses

Agent quality rarely crashes, it drifts. Here's how to instrument agents, read the signals early, and set thresholds that catch decay before users do.

Matt Doughty
All posts 95 articles
The Offline-to-Online Evaluation Gap: Why Your Agent Tests Pass but Production Fails
Evaluation

The Offline-to-Online Evaluation Gap: Why Your Agent Tests Pass but Production Fails

Offline agent evaluations miss 30-40% of real-world failure modes. Here is how to layer trace-based and online evaluation to catch what test sets cannot.

Matt Doughty
Why AI Agents Fail Production Evaluation: The Reliability Gap Between Benchmarks and Real-World Deployment
Evaluation

Why AI Agents Fail Production Evaluation: The Reliability Gap Between Benchmarks and Real-World Deployment

Production AI agents fail at rates between 70, 95% depending on task complexity. Here is what evaluation frameworks miss and what to measure instead.

Matt Doughty
The Silent Failure Gap: Why 88% of AI Agents Fail Quality Evaluation Before Production Reaches Scale
Evaluation

The Silent Failure Gap: Why 88% of AI Agents Fail Quality Evaluation Before Production Reaches Scale

88% of AI agent projects fail before production scale. Here is what the measurement gap looks like and how to close it before deployment compounds the cost.

Matt Doughty
Measuring what agents actually cost: hidden token overhead and efficiency gaps
Evaluation

Measuring what agents actually cost: hidden token overhead and efficiency gaps

A 4x token cost gap between identical agent outputs shows why instrumentation matters before you scale. Here is how to measure and fix it.

Matt Doughty
Evals vs Observability: Watching Your Agents Is Not Evaluating Them
MCP

Evals vs Observability: Watching Your Agents Is Not Evaluating Them

Observability tells you what your agent did. Evaluation tells you whether it was any good. Here is why you need both, and how to wire them together.

Matt Doughty
Agent Evaluation in Production: What to Measure and How to Prove It
Evaluation

Agent Evaluation in Production: What to Measure and How to Prove It

A practitioner playbook for measuring AI agent quality in production: task success rate, drift, failure modes, and building an evidence chain from spans to sc

Matt Doughty
Ghost Actions: When Your AI Agent Does Things Nobody Asked For
Evaluation

Ghost Actions: When Your AI Agent Does Things Nobody Asked For

Ghost actions are agent behaviors nobody requested. Here is what they look like, why standard monitoring misses them, and how to catch them before they cause

Matt Doughty
What Customers Ask Before They Trust Your AI Agent
Evaluation

What Customers Ask Before They Trust Your AI Agent

The five questions enterprise buyers ask before they'll rely on your AI agent, and the evidence artifacts that answer each one before a deal stalls.

Matt Doughty
Product releases

5 Questions Every Head of AI Should Ask About Agent Governance

Scaling AI agents from pilots to production requires governance infrastructure. These five questions help Heads of AI evaluate whether their organisation can scale agents responsibly.

Matt Doughty
Product releases

5 Questions Every ML Engineer Should Ask About Agent Runtime Controls

ML engineers building AI agents need runtime controls that work with their development workflow — not against it. These five questions help evaluate agent governance from an engineering perspective.

Matt Doughty
Product releases

5 Questions Every AI Product Manager Should Ask About Agent Governance

AI product managers must balance user experience with governance requirements. These five questions help PMs ship agent-powered products that are both useful and responsible.

Matt Doughty
Product releases

5 Questions Every Risk Manager Should Ask About AI Agent Deployments

AI agents introduce risk categories that traditional risk frameworks do not cover. These five questions help risk managers evaluate and mitigate the unique risks of autonomous AI agents.

Matt Doughty
Product releases

5 Questions Every CISO Should Ask Before Deploying AI Agents

AI agents introduce attack surfaces that traditional security tools were not designed for. These five questions help CISOs evaluate whether their organisation is ready to deploy agents safely.

Matt Doughty
Product releases

5 Questions Every AI Governance Lead Should Ask About Agent Oversight

AI governance frameworks designed for models do not cover agents. These five questions help governance leads extend their programmes to address the unique challenges of autonomous AI agents.

Matt Doughty
Why did I join Prefactor by Josh Gillies
About Prefactor

Why did I join Prefactor by Josh Gillies

Prefactor's new engineer explains why he joined Prefactor

Josh Gillies
Taming the Lobster: Announcing Prefactor’s Integration with OpenClaw
Product releases

Taming the Lobster: Announcing Prefactor’s Integration with OpenClaw

Prefactor announces a new integration with OpenClaw (Clawdbot)

Matt Doughty
AI Model Watermarking for Enterprise Security
Security

AI Model Watermarking for Enterprise Security

How cryptographic and forensic watermarks embedded in AI models and outputs help enterprises prove ownership, detect misuse, and meet compliance.

Matt Doughty
How to Analyze Multi-Agent AI Attack Surfaces
Security

How to Analyze Multi-Agent AI Attack Surfaces

Framework to inventory agents, map dependencies, detect context poisoning and prompt injection, and apply behavioral and static analysis to secure multi-agent AI.

Matt Doughty
Best Practices for MCP Audit Compliance
MCP

Best Practices for MCP Audit Compliance

Secure MCP agent access with least-privilege controls, tamper-proof audit trails, automated access reviews, real-time monitoring, and permission fixes.

Matt Doughty
AI Agent Identity Audits: Reporting Standards
Agent Identity

AI Agent Identity Audits: Reporting Standards

Standards for auditing AI agent identities, metrics, and reports to ensure traceability, verified human ownership, and compliance with HIPAA, SOX, and GDPR.

Matt Doughty
MCP Breach Detection Best Practices
MCP

MCP Breach Detection Best Practices

Secure MCP systems with detailed logging, EDR and AI behavioral analytics, protocol validation, centralized audit trails, and rapid containment controls.

Matt Doughty
Best CI/CD Tools for MCP Integration
MCP

Best CI/CD Tools for MCP Integration

Compare GitHub Actions, GitLab CI, Azure DevOps, Jenkins, and cloud-native CI for secure MCP integration and agent governance with Prefactor.

Matt Doughty
MAESTRO Framework: Threat Modeling for AI Agents
Security

MAESTRO Framework: Threat Modeling for AI Agents

MAESTRO maps AI agent security into seven layers to identify and mitigate adversarial attacks, data poisoning, impersonation, and runtime threats.

Matt Doughty
MCP MFA Compliance Checklist
MCP

MCP MFA Compliance Checklist

Practical checklist to enforce phishing-resistant MFA, secure AI agent identities, apply RBAC, and log/audit MCP access for regulatory compliance.

Matt Doughty
Top Features of AI Vulnerability Scanning Tools
Security

Top Features of AI Vulnerability Scanning Tools

Key features of AI vulnerability scanners: real-time monitoring, AI-specific threat detection, CI/CD and MLOps integration, governance and scalable fixes.

Matt Doughty
How MCP Secures Agent Authentication Compliance
MCP

How MCP Secures Agent Authentication Compliance

How MCP uses OAuth 2.1 with PKCE, resource indicators, scoped tokens, and audit trails to enforce least privilege and meet regulatory requirements.

Matt Doughty
Data Retention for AI Agents in Regulated Industries
Compliance

Data Retention for AI Agents in Regulated Industries

Guidance on AI agent log retention across healthcare, finance, and EU/US law—recommended retention periods, privacy controls, and centralized compliance practices.

Matt Doughty
Best Practices for Agent-to-Agent Authentication
Authentication

Best Practices for Agent-to-Agent Authentication

Secure AI agent interactions with unique identities, short-lived tokens, mTLS, OAuth client credentials, and continuous monitoring for audit and compliance.

Matt Doughty
Audit Trails in CI/CD: Best Practices for AI Agents
Compliance

Audit Trails in CI/CD: Best Practices for AI Agents

Checklist to build immutable, centralized audit trails in CI/CD that trace AI agent actions, ensure regulatory compliance, and speed incident response.

Matt Doughty
Securing AI Agents with Role-Based Delegation
Access Control

Securing AI Agents with Role-Based Delegation

Secure AI agents with scoped, short-lived roles and RFC 8693 delegation tokens, enforcing least privilege, RBAC+ABAC, audit trails, and centralized governance for compliance.

Matt Doughty
MCP Security: Dynamic Authorization Explained
MCP

MCP Security: Dynamic Authorization Explained

How MCP uses OAuth 2.1, resource indicators, and short-lived scoped tokens to give AI agents fine-grained, auditable access while supporting compliance.

Matt Doughty
How MCP Secures Human-to-Agent Delegation
MCP

How MCP Secures Human-to-Agent Delegation

Tie AI agent actions to verified users with scoped, short-lived tokens, audit trails, and HITL approvals to prevent over-permissioning and token misuse.

Matt Doughty
MCP Security for Multi-Tenant AI Agents: Explained
MCP

MCP Security for Multi-Tenant AI Agents: Explained

Secure multi-tenant AI agents with MCP using tenant-specific IDs, short-lived tokens, encryption, and audit trails; covers isolation, auth, and governance.

Matt Doughty
How MCP Enhances Audit Trails for Agent Authentication
MCP

How MCP Enhances Audit Trails for Agent Authentication

How MCP gives AI agents unique identities and uses OAuth 2.1+PKCE while Prefactor adds real-time, context-rich audit trails for compliance.

Matt Doughty
Ultimate Guide to Non-Human Identity Risk Mitigation
Agent Identity

Ultimate Guide to Non-Human Identity Risk Mitigation

How to inventory, secure, rotate, and monitor machine identities—API keys, service accounts, and AI agents—to enforce least privilege and reduce breach risk.

Matt Doughty
PKCE in OAuth for AI Agents: Best Practices
Authentication

PKCE in OAuth for AI Agents: Best Practices

Guide to PKCE for AI agents: generate S256 verifiers, enforce PKCE server-side, use short scoped tokens, validate redirects, and monitor PKCE flows.

Matt Doughty
Regulatory Standards for AI Agent Identity
Compliance

Regulatory Standards for AI Agent Identity

Assign cryptographic identities to AI agents, enforce time-limited least-privilege access, and maintain auditable logs to meet GDPR, HIPAA, and NIST requirements.

Matt Doughty
Real-Time Agent Logging with MCP
MCP

Real-Time Agent Logging with MCP

Structured JSON logs, correlation IDs, and Prefactor audit trails for secure, real-time agent monitoring, debugging, and compliance.

Matt Doughty
How MCP Enhances AI Agent Security in Multi-Cloud
MCP

How MCP Enhances AI Agent Security in Multi-Cloud

Standardize AI agent identity, scoped tokens, and real-time policy enforcement across AWS, Azure, and GCP; Prefactor automates token workflows and audit trails.

Matt Doughty
How MCP Secures Agent Identity Lifecycle
MCP

How MCP Secures Agent Identity Lifecycle

Secure AI agent identities with MCP and Prefactor using OAuth/OIDC, scoped provisioning, automated credential rotation, continuous monitoring, and instant revocation.

Matt Doughty
AI Agent Identity Lifecycle: Best Practices
Agent Identity

AI Agent Identity Lifecycle: Best Practices

Treat AI agents as first-class identities: enforce least-privilege provisioning, short-lived tokens, CI/CD automation, continuous monitoring, and secure deprovisioning.

Matt Doughty
Granular Access Control with MCP
MCP

Granular Access Control with MCP

How MCP uses OAuth 2.1, scoped tokens, and policy-as-code to enforce least-privilege access for AI agents, multi-tenant apps, and CI/CD workflows.

Matt Doughty
CI/CD Integration for AI Agents: Q&A
Developer Experience

CI/CD Integration for AI Agents: Q&A

Explore the complexities and security challenges of integrating AI agents into CI/CD pipelines, along with best practices for effective management.

Matt Doughty
Ultimate Guide to Multi-Tenant AI Systems
Access Control

Ultimate Guide to Multi-Tenant AI Systems

Explore the complexities of multi-tenant AI systems, focusing on security, identity management, and compliance challenges.

Matt Doughty
How to Secure MCP Servers with OAuth 2.1 in FastAPI
MCP

How to Secure MCP Servers with OAuth 2.1 in FastAPI

Learn how to implement OAuth 2.1 authentication for MCP servers in FastAPI. Step-by-step guide for securing remote AI applications.

Matt Doughty
Model Context Protocol: Setup and Implementation
MCP

Model Context Protocol: Setup and Implementation

Learn how to implement the Model Context Protocol for secure, automated authentication between AI agents and systems, enhancing compliance and efficiency.

Matt Doughty
Solving AI Agent Scalability Issues
Security

Solving AI Agent Scalability Issues

Explore effective strategies for managing the identity lifecycle of AI agents, ensuring security, compliance, and scalability in dynamic environments.

Matt Doughty
AI Agent Security Checklist for CTOs
Security

AI Agent Security Checklist for CTOs

Explore essential security strategies for AI agents, focusing on identity management, authentication, risk controls, and compliance.

Matt Doughty
How to Secure AI Agent Authentication in 2025
Authentication

How to Secure AI Agent Authentication in 2025

Explore essential strategies for securing AI agent authentication in 2025, focusing on unique credentials, JIT access, and compliance standards.

Matt Doughty
5 Best Practices for AI Agent Access Control
Access Control

5 Best Practices for AI Agent Access Control

Explore essential practices for securing AI agents, focusing on identity management, access control, and the importance of human oversight.

Matt Doughty
How to Build Custom Consent Screens for AI Agents Handling Sensitive Data
Guides

How to Build Custom Consent Screens for AI Agents Handling Sensitive Data

Learn how to build sophisticated consent screens that explain AI agent actions clearly. Discover advanced authorization patterns beyond basic authentication

Matt Doughty
How to Handle Dynamic Client Registration for AI Agents That Spawn and Terminate Automatically
Authentication

How to Handle Dynamic Client Registration for AI Agents That Spawn and Terminate Automatically

Learn why AI agents need device-like Dynamic Client Registration, not application-style permanent registration. Discover how Prefactor's DCR handles ephemeral agent lifecycles automatically.

Matt Doughty
How to Build a Security-First MCP Architecture: Design Patterns and Implementation
MCP

How to Build a Security-First MCP Architecture: Design Patterns and Implementation

Architectural patterns for building inherently secure MCP systems, including zero-trust principles, defense in depth, and secure by design approaches for AI agents.

Matt Doughty
Why Traditional API Security Fails with MCP and What to Do Instead
MCP

Why Traditional API Security Fails with MCP and What to Do Instead

Analysis of why conventional API security approaches don't work for MCP, and new security paradigms needed for AI agent architectures and autonomous systems.

Matt Doughty
How to Secure Third-Party MCP Integrations: Atlassian, Linear, and Canva
MCP

How to Secure Third-Party MCP Integrations: Atlassian, Linear, and Canva

Security framework for popular MCP integrations including Atlassian MCP, Linear MCP, and Canva MCP, covering API security and data protection strategies.

Matt Doughty
What Security Controls Should You Implement for Enterprise MCP Deployments?
MCP

What Security Controls Should You Implement for Enterprise MCP Deployments?

Enterprise-grade security checklist covering network security, data governance, compliance requirements, and audit trails for large-scale MCP deployments.

Matt Doughty
How to Secure Claude Code MCP Integrations in Production
MCP

How to Secure Claude Code MCP Integrations in Production

How to secure Claude Code MCP integrations in production with scoped access, runtime controls, and auditable tool permissions.

Matt Doughty
Where MCP Security Breaks: Common Attack Vectors and Prevention
MCP

Where MCP Security Breaks: Common Attack Vectors and Prevention

Analysis of common MCP attack patterns including prompt injection, privilege escalation, and data poisoning, with prevention strategies.

Matt Doughty
Why MCP Inspector is Essential for Security Testing and Validation
MCP

Why MCP Inspector is Essential for Security Testing and Validation

Deep dive into using MCP Inspector for security testing, vulnerability discovery, and protocol validation, with practical testing scenarios.

Matt Doughty
What Are the Critical MCP Security Risks Every Developer Must Know?
MCP

What Are the Critical MCP Security Risks Every Developer Must Know?

Model Context Protocol (MCP) introduces unique security challenges that traditional API security doesn't address.

Matt Doughty
Claude x Canva Remote MCP server demo
Product releases

Claude x Canva Remote MCP server demo

Live demo showing how to connect Cursor to Canva using Remote MCP — enabling secure agent access across tools.

Matt Doughty
Prefactor x Claude Remote MCP server demo
Product releases

Prefactor x Claude Remote MCP server demo

Live demo showing how to connect Cursor to Canva using Remote Model Context Protocol (MCP) — enabling secure, agent-driven access between tools.

Matt Doughty
MCP vs AI Agents: What’s the Difference?
MCP

MCP vs AI Agents: What’s the Difference?

Learn the difference between Model Context Protocol (MCP) and AI agents — and how MCP provides the access and security layer that agents need to function safely.

Matt Doughty
What Is an MCP Gateway?
MCP

What Is an MCP Gateway?

Learn what an MCP Gateway is, how it fits into the Model Context Protocol stack, and how it simplifies secure agent access to APIs.

Matt Doughty
What’s the Difference Between an MCP Server and MCP Client?
MCP

What’s the Difference Between an MCP Server and MCP Client?

Understand the difference between MCP servers and MCP clients — and how they work together to enable secure access for AI agents and automated systems.

Matt Doughty
Top 10 MCP Security Risks (and How to Avoid Them)
MCP

Top 10 MCP Security Risks (and How to Avoid Them)

MCP (Model Context Protocol) opens the door to powerful agentic AI — but also introduces serious security risks. Here are the top 10 vulnerabilities in MCP deployments, and how your team can defend against them.

Matt Doughty
What Is MCP — and Why Is Everyone Talking About It?
MCP

What Is MCP — and Why Is Everyone Talking About It?

Learn what Model Context Protocol (MCP) is, why it's suddenly everywhere, and what it means for the future of AI agents, APIs, and secure access.

Matt Doughty
MCP vs LLM: What’s the Difference?
MCP

MCP vs LLM: What’s the Difference?

Understand the difference between Model Context Protocol (MCP) and Large Language Models (LLMs) — and how they interact in AI-powered systems.

Matt Doughty
What’s the Difference Between MCP and an API?
MCP

What’s the Difference Between MCP and an API?

Understand how Model Context Protocol (MCP) differs from traditional APIs — and why it matters for agent-based access, identity, and control.

Matt Doughty
Top 10 Agent Integrations to Add to Your SaaS
Agent Identity

Top 10 Agent Integrations to Add to Your SaaS

Ten high-value agent integrations that show where identity, delegation, and runtime control start to matter for SaaS teams.

Matt Doughty
Prefactor: MCP Auth, Agent Identity, and the Future of Authentication
About Prefactor

Prefactor: MCP Auth, Agent Identity, and the Future of Authentication

Prefactor is building agent-first authentication infrastructure for AI-native platforms. Learn how we support MCP, secure agent identity, and our vision for programmable, portable access across autonomous systems.

Matt Doughty
How to Implement MCP Authentication (Step-by-Step Guide for SaaS apps)
MCP

How to Implement MCP Authentication (Step-by-Step Guide for SaaS apps)

Learn how to implement MCP authentication for AI agents using scoped delegation, agent identity, and signed access tokens. This guide covers token generation, validation, and best practices for MCP login support in SaaS and AI-native apps.

Matt Doughty
Beyond the Prompt: Securing Agent Behavior, Not Just Access
Security

Beyond the Prompt: Securing Agent Behavior, Not Just Access

Securing agent behaviour

Matt Doughty
The Compliance Conundrum: Auditing Autonomous Agent Actions
Compliance

The Compliance Conundrum: Auditing Autonomous Agent Actions

Auditing Autonomous Agent Actions

Matt Doughty
Security Risks in the Age of Autonomous Agents: Beyond Traditional Secrets Management
Security

Security Risks in the Age of Autonomous Agents: Beyond Traditional Secrets Management

Beyond secrets management

Matt Doughty
How to Design Identity for AI Agents, Not Just Humans and APIs
Agent Identity

How to Design Identity for AI Agents, Not Just Humans and APIs

A practical framework for designing identity around AI agents, delegated access, and runtime accountability.

Matt Doughty
Why M2M Tokens Aren’t Enough for Agent-Based Systems: Beyond Static Credentials
Authentication

Why M2M Tokens Aren’t Enough for Agent-Based Systems: Beyond Static Credentials

M2M tokens aren't enough

Matt Doughty
Service Accounts Are Failing in the Age of Agent Identity
Agent Identity

Service Accounts Are Failing in the Age of Agent Identity

Why service accounts break down for AI agents, and what an agent-first identity model needs to support.

Matt Doughty
From Static to Dynamic: What Agent Identity Actually Looks Like
Agent Identity

From Static to Dynamic: What Agent Identity Actually Looks Like

What dynamic agent identity looks like in production, from short-lived credentials to revocation and auditability.

Matt Doughty
Zero Trust for Agents: What It Actually Looks Like
Security

Zero Trust for Agents: What It Actually Looks Like

Zero Trust for Agents

Matt Doughty
Designing a DSL for Agent Access Control
Access Control

Designing a DSL for Agent Access Control

Why agent access control needs a policy language teams can version, review, and enforce across runtimes.

Matt Doughty
Impersonation ≠ Delegation: Don’t Let Agents Spoof Your Users
Access Control

Impersonation ≠ Delegation: Don’t Let Agents Spoof Your Users

How to stop AI agents from spoofing user identity by enforcing explicit delegation, scoped access, and auditable actions.

Matt Doughty
Agent Identity 101: Why Naming, Scoping, and Lifecycle Matter
Agent Identity

Agent Identity 101: Why Naming, Scoping, and Lifecycle Matter

A practical introduction to agent identity: naming, scoping, ownership, and lifecycle controls for non-human actors.

Matt Doughty
How to Secure Agents Acting on Behalf of Users
Security

How to Secure Agents Acting on Behalf of Users

Securing agents acting as humans

Matt Doughty
Autonomous Agents Create a New Identity Challenge
Agent Identity

Autonomous Agents Create a New Identity Challenge

Why autonomous agents break assumptions behind traditional identity systems, and what teams need instead.

Matt Doughty
Your Customers' AI Agents Need to Log In—Is Your SaaS Ready?
Agent Identity

Your Customers' AI Agents Need to Log In—Is Your SaaS Ready?

Is your saas ready for agents?

Matt Doughty
Compare Agent Authentication Solutions
Comparisons

Compare Agent Authentication Solutions

How to compare agent authentication platforms by delegation, runtime control, auditability, and developer ergonomics.

Matt Doughty
What Most Companies Get Wrong About Non-Human Identity Management
Agent Identity

What Most Companies Get Wrong About Non-Human Identity Management

Common mistakes teams make with non-human identities, and what changes when AI agents become first-class actors.

Matt Doughty
How to Manage Non-Human Identities Before They Manage You
Agent Identity

How to Manage Non-Human Identities Before They Manage You

How to manage non-human identities with ownership, scoping, rotation, and lifecycle controls once AI agents enter production.

Matt Doughty
What Are Non-Human Identities in AI Agent Systems?
Agent Identity

What Are Non-Human Identities in AI Agent Systems?

A clear definition of non-human identities, how they differ from service accounts and bots, and why AI agents make them harder to manage.

Matt Doughty
How to Secure AI-Assisted Vibe-Coded Applications
Reports

How to Secure AI-Assisted Vibe-Coded Applications

The main risks in AI-assisted vibe-coded applications, plus practical controls for authentication, authorization, and agent safety.

Matt Doughty
Why build Prefactor now?
About Prefactor

Why build Prefactor now?

It's a crowded space. Why now?

Matt Doughty
Our Vision for Developer Experience at Prefactor
About Prefactor

Our Vision for Developer Experience at Prefactor

How we're approaching developer experience in user security through our domain-specific language

Matt Doughty
Prefactor DSL: The Unified Language for Authentication, Authorization, and Audit
Product releases

Prefactor DSL: The Unified Language for Authentication, Authorization, and Audit

A powerful DSL that transforms fragmented user security into a cohesive, code-first platform for developers who need precision, control, and simplicity.

Matt Doughty
The Story Behind Prefactor: Built by Founders Who Lived the Problem
About Prefactor

The Story Behind Prefactor: Built by Founders Who Lived the Problem

How Prefactor came to life direct from the founders

Matt Doughty

See how every agent performs — and make it better

Prefactor helps teams observe, evaluate, and improve their AI agents in production — across every framework and provider.

Book a demo View docs