Insights & research

The Prefactor Blog

Field notes on running AI agents in production: governance, security, identity, and the evidence trail that turns a working demo into something you can trust.

Tool-Use Hallucination: Detecting When Agents Fake Execution Without Triggering Alerts
Evaluation

Tool-Use Hallucination: Detecting When Agents Fake Execution Without Triggering Alerts

Tool-use hallucination lets agents report successful tool calls that never ran, sending bad parameters or inventing results while task metrics show green.

Matt Doughty
All posts99 articles
Why 61% of AI Agents Pass Evaluations But Fail Customers: Production-Grade Evaluation Metrics That Actually Predict Failure
Evaluation

Why 61% of AI Agents Pass Evaluations But Fail Customers: Production-Grade Evaluation Metrics That Actually Predict Failure

61% of enterprise AI agents pass internal evals then fail customers. Here are the metrics that actually predict production failure before deployment.

Matt Doughty
Reliable Single vs. Multi-Step: How to Evaluate Agent Consistency Across Repeated Runs
Evaluation

Reliable Single vs. Multi-Step: How to Evaluate Agent Consistency Across Repeated Runs

Single-run benchmarks miss the consistency failures that matter in production. Here is how to measure pass@k and catch variance before it reaches users.

Matt Doughty
Silent Pass, Wrong Answer: The Gap Between Agent Uptime Monitoring and Behavioral Correctness
Evaluation

Silent Pass, Wrong Answer: The Gap Between Agent Uptime Monitoring and Behavioral Correctness

Agents can answer with confidence, pass every health check, and still be wrong. Here is what uptime monitoring misses and how behavioral evaluation fills the

Matt Doughty
Three incidents in 141,006 runs: why sampling misses them
Evaluation

Three incidents in 141,006 runs: why sampling misses them

Anthropic found 3 incidents in 141,006 eval runs. A 5% sample expects 0.15 hits. The math shows why sampling concludes "all clear" when it shouldn't.

Matt Doughty
An agent cancelled a stranger's gym class to get its user in
Evaluation

An agent cancelled a stranger's gym class to get its user in

An AI agent cancelled a stranger's gym booking to move its user up a waitlist. The API had no authorisation checks. The agent passed every obvious success met

Matt Doughty
Every Felony Bench incident happened inside an evaluation
Evaluation

Every Felony Bench incident happened inside an evaluation

Every incident on Felony Bench happened inside a run someone called "evaluation." The environment label changed nothing; the tools were real.

Matt Doughty
Gemini hacked three companies and the test still passed
Evaluation

Gemini hacked three companies and the test still passed

Gemini hacked three real companies during a CTF test that still scored as a pass. Here is why result-only scoring misses what agents actually do.

Matt Doughty
OpenAI's five agent failure patterns: task done, method wrong
Evaluation

OpenAI's five agent failure patterns: task done, method wrong

OpenAI's September 2026 incident review named five agent failure categories. Here is what each one looks like in an ordinary product agent you have already sh

Matt Doughty
OpenAI found the Medicare hack in its logs 54 days later
Evaluation

OpenAI found the Medicare hack in its logs 54 days later

OpenAI found an unauthorized Medicare data access 54 days after it happened. Here is what evaluating each run as it runs would have surfaced instead.

Matt Doughty
What Vercel and Bryo AI's Jev results show, and what they do not
Evaluation

What Vercel and Bryo AI's Jev results show, and what they do not

Vercel and Bryo AI report faster, cheaper results with Jev. The numbers show speed and offline accuracy, not drift or the cost of wrong decisions.

Matt Doughty
Do decision models like Jev need evals? Why silent steps matter
Evaluation

Do decision models like Jev need evals? Why silent steps matter

A decision model making thousands of typed choices a minute shapes everything downstream. Its silent mistakes compound, so it needs more scrutiny, not less.

Matt Doughty
Is Jev's confidence score trustworthy? RLCD calibration explained
Evaluation

Is Jev's confidence score trustworthy? RLCD calibration explained

Jev's confidence score is calibrated on TypeSafe's training data, not your traffic. Until you compare it to real outcomes per run, you cannot act on it.

Matt Doughty
Did swapping an LLM step for Jev make your agent worse?
Evaluation

Did swapping an LLM step for Jev make your agent worse?

Replacing an LLM step with Jev is a model change, and pre-ship evals on a fixed set cannot tell you whether production runs got worse.

Matt Doughty
Does Jev model routing need evaluation? The unmeasured failure
Evaluation

Does Jev model routing need evaluation? The unmeasured failure

A Jev router that sends a hard request to a cheap model logs a clean 200 and a cost saving. The wrong answer lands on the customer, not in your logs.

Matt Doughty
Can Jev typed outputs still be wrong? Type safety vs correctness
Evaluation

Can Jev typed outputs still be wrong? Type safety vs correctness

Jev guarantees a valid instance of the requested type, not a correct decision. Typed outputs still require behaviour evals to catch semantic errors.

Matt Doughty
What is Jev? Why a model that cannot hallucinate can still fail
Evaluation

What is Jev? Why a model that cannot hallucinate can still fail

Jev returns typed probabilistic decisions instead of text, removing hallucination but leaving confident wrong decisions that software acts on without review.

Matt Doughty
Canary evaluation for production AI agents: a practical guide
Evaluation

Canary evaluation for production AI agents: a practical guide

Silent quality drift in production AI agents is detectable within 24 to 48 hours using canary evaluation. Infrastructure metrics alone cannot catch it.

Matt Doughty
AI agent audit trails: what to capture and how to use it
Evaluation

AI agent audit trails: what to capture and how to use it

Span records, not prompt logs, are what make AI agent investigations possible. Each field you skip is a gap you cannot close after the fact.

Matt Doughty
How to benchmark AI agents across model versions
Evaluation

How to benchmark AI agents across model versions

Stable benchmarks require fixed inputs, defined success criteria, and consistent instrumentation. Varying only one factor at a time makes results comparable.

Matt Doughty
Multi-agent observability: spans, scores, and schema validation
Agent Maturity Curve

Multi-agent observability: spans, scores, and schema validation

Cross-boundary spans, per-agent quality scores, and schema validation isolate failures in multi-agent systems before they compound across the chain.

Matt Doughty
How cascading failure compounds across multi-step agent workflows
Evaluation

How cascading failure compounds across multi-step agent workflows

A cascade starts when one step returns a plausible wrong output and later steps build on it. Boundary validation and checkpoints contain it; retries do not.

Matt Doughty
How to detect silent failures in AI agent responses
Evaluation

How to detect silent failures in AI agent responses

Empty responses and silent tool failures return HTTP 200 and close spans normally. Span-level output validation catches what status checks miss.

Matt Doughty
How to detect agent misbehavior in production at scale
Evaluation

How to detect agent misbehavior in production at scale

Behavior deviation detection, reasoning artifact capture, and adversarial prompt detection run continuously so human reviewers act on signals, not haystacks.

Matt Doughty
How to detect unauthorized AI agent actions in production
Evaluation

How to detect unauthorized AI agent actions in production

Span-based instrumentation and activity schemas surface prompt injection, unauthorized access, and multi-turn drift before incidents compound.

Matt Doughty
How to evaluate agent behavior under model jailbreaks
Evaluation

How to evaluate agent behavior under model jailbreaks

Activity schemas and validation suites let you catch policy violations and capability regressions in AI agents before they reach production.

Matt Doughty
Step-level evaluation in production agents: what to measure
Evaluation

Step-level evaluation in production agents: what to measure

Final-output benchmarks hide compounding failures. Instrumenting each step shows where accuracy drops and why workflows fail at scale.

Matt Doughty
Multi-agent coordination failures: why per-agent evals miss them
Evaluation

Multi-agent coordination failures: why per-agent evals miss them

Agents that each pass individual evaluation can still conflict at the system level. Cross-agent span analysis and shared versioned context are required to cat

Matt Doughty
Why AI agents pass evals but fail in production
Evaluation

Why AI agents pass evals but fail in production

Outcome metrics miss step-level failures that compound across runs, creating a 35-point gap between benchmark scores and production success rates.

Matt Doughty
How to stop AI agent quality decay before alerts fire
Evaluation

How to stop AI agent quality decay before alerts fire

Silent output degradation is the leading cause of cancelled agent projects. Score per span, baseline tool calls, and trace full paths to catch it early.

Matt Doughty
Step-level agent evaluation: measure every intermediate step
Evaluation

Step-level agent evaluation: measure every intermediate step

At 95% per-step accuracy, a 10-step agent succeeds only 59% of the time. Step-level spans show which step fails and why.

Matt Doughty
Why multi-step agents fail in production but pass benchmarks
Evaluation

Why multi-step agents fail in production but pass benchmarks

Step repetitions cause 17% of multi-agent failures and reasoning-to-action mismatches cause 14%, both undetected by final-output evaluation.

Matt Doughty
Why 85% step accuracy collapses to 20% end-to-end success
Evaluation

Why 85% step accuracy collapses to 20% end-to-end success

At 85% per-step accuracy, a ten-step agent completes the whole task about one time in five. Final-output scoring cannot see this; trajectory evaluation can.

Matt Doughty
Why 88% of agent pilots fail to reach production
Agent Maturity Curve

Why 88% of agent pilots fail to reach production

88% of agent pilots never reach production. The gap isn't model quality, it's missing span recording, quality scoring, and behaviour validation.

Matt Doughty
AI agent evaluation baseline: what to record before launch
Evaluation

AI agent evaluation baseline: what to record before launch

A pre-launch baseline records process cost, quality rate, and cycle time so agent gains can be separated from noise. It does not guarantee ROI.

Matt Doughty
Why multi-agent orchestration fails silently in production
Evaluation

Why multi-agent orchestration fails silently in production

79% of multi-agent failures come from coordination gaps, not model quality. Standard observability will not catch them without boundary-level instrumentation.

Matt Doughty
Per-turn evaluation: scoring every agent step at runtime
Evaluation

Per-turn evaluation: scoring every agent step at runtime

Per-turn evaluation scores each agent step at runtime, catching the wrong retrievals and tool errors that final-outcome benchmarks miss in 67% of production f

Matt Doughty
Why multi-step agent workflows need step-level failure diagnosis
Evaluation

Why multi-step agent workflows need step-level failure diagnosis

Final-output scores miss most multi-step agent failures. Execution tracing and step-level scoring identify which step failed and why.

Matt Doughty
Coding agent speed vs accuracy: how to decide
Comparisons

Coding agent speed vs accuracy: how to decide

Accuracy-optimized agents lower defect rates; speed-optimized ones cut latency. The right choice depends on deployment stage, cascade risk, and cost per accep

Matt Doughty
Span-level instrumentation for AI agent cost spikes
Evaluation

Span-level instrumentation for AI agent cost spikes

Span-level instrumentation attributes every model call to a specific agent, step, and token count so cost spikes become correctable events, not unexplained nu

Matt Doughty
How to detect agent behavioral drift using span-level traces
Evaluation

How to detect agent behavioral drift using span-level traces

Span instrumentation records every agent decision. Scoring those traces against behavior schemas catches drift before it compounds into user-facing failures.

Matt Doughty
Why production agents succeed only 56.6% of the time
Evaluation

Why production agents succeed only 56.6% of the time

Agents score 90%+ on benchmarks but succeed 56.6% of the time in production. Up to 75% of those failures report success instead of failure.

Matt Doughty
MCP schema drift: detecting parameter changes before agent calls fail
Evaluation

MCP schema drift: detecting parameter changes before agent calls fail

MCP tool parameter schemas change independently of descriptions, leaving agents acting on contracts the server no longer accepts.

Matt Doughty
How to assess AI agent readiness before production deployment
Agent Maturity Curve

How to assess AI agent readiness before production deployment

Agent readiness depends on four maturity indicators and four evaluation gates, not model quality or pilot performance alone.

Matt Doughty
How to measure coding agent code quality with session analysis
Evaluation

How to measure coding agent code quality with session analysis

Instrument agent sessions to capture tool call sequences and replay them after execution. Merge rate alone misses defects that tests and reviewers skip.

Matt Doughty
DAG-structured evaluation for multi-step agent failures
Evaluation

DAG-structured evaluation for multi-step agent failures

Cascade failures originate upstream, not at the final step. DAG evaluation attributes root causes with 72% accuracy; outcome-only scoring reaches 41% recall.

Matt Doughty
Why agents pass benchmarks but fail in production
Evaluation

Why agents pass benchmarks but fail in production

Benchmark scores miss 37% of production agent failures because they grade terminal answers, not trajectories. Step-level metrics are what close that gap.

Matt Doughty
How to measure coding agent tool call quality, not just speed
Evaluation

How to measure coding agent tool call quality, not just speed

1Password found 74% of AI-generated patches fail correctness checks. Scoring tool call spans catches what completion time cannot.

Matt Doughty
Why benchmark success rates drop 36 points in production
Evaluation

Why benchmark success rates drop 36 points in production

Agents score 47% on offline benchmarks but 11% in production. Step-level accuracy and faithfulness scores close that gap.

Matt Doughty
Activity schema validation: how to catch agent drift early
Evaluation

Activity schema validation: how to catch agent drift early

Span-level validation checks each agent action against a declared behavioral contract, catching unauthorized tool use before credential theft or privilege esc

Matt Doughty
Step-level evaluation for AI agents: a practical guide
Evaluation

Step-level evaluation for AI agents: a practical guide

Step-level evaluation scores each tool call and reasoning transition individually, exposing where agent failures begin before they reach the final output.

Matt Doughty
How to validate agent behavior against intent before deployment
Evaluation

How to validate agent behavior against intent before deployment

Span-level validation checks each step in an agent's execution trace against an activity schema, catching tool misuse and sequencing errors before release.

Matt Doughty
Trajectory evaluation: what final-answer scoring hides in agent runs
Evaluation

Trajectory evaluation: what final-answer scoring hides in agent runs

Agents scored only on output pass 20 to 40% more test cases than step-level review reveals, hiding fabricated steps and fragile reasoning paths.

Matt Doughty
How to measure agentic workflow costs beyond token pricing
Agent Maturity Curve

How to measure agentic workflow costs beyond token pricing

Token prices fell 67% yet most enterprises still overspent on AI. Iteration count, model switching, and tool call overhead are the real cost drivers.

Matt Doughty
Why AI agents pass evals but fail silently at scale
Evaluation

Why AI agents pass evals but fail silently at scale

Agents that score well on benchmarks still fail without raising errors in production. Understanding why requires measuring quality drift continuously.

Matt Doughty
How to find the root cause of agent failures in execution traces
Evaluation

How to find the root cause of agent failures in execution traces

Structured traces let you pinpoint which step failed and why, but only if each span records inputs, tool status, model version, and retrieval context.

Matt Doughty
Why AI agents stall between deployment and production scale
Agent Maturity Curve

Why AI agents stall between deployment and production scale

97% of companies have deployed AI agents, but only 11% run them at scale. Span instrumentation, schema validation, and incident scoring close that gap.

Matt Doughty
How to measure agent token spend at the span level
Evaluation

How to measure agent token spend at the span level

Attach cost figures to each tool call and planning loop, build baselines by agent type, and separate normal variance from runaway behavior before bills grow.

Matt Doughty
Agent capability boundaries: where models fail and why
Agent Maturity Curve

Agent capability boundaries: where models fail and why

Agents fail at predictable capability boundaries tied to task duration, tool access, and production load. Knowing the stage tells you what to test.

Matt Doughty
Why most AI agent pilots never reach production
Evaluation

Why most AI agent pilots never reach production

Around 88% of enterprise AI agent pilots stall before production. The gap is not model quality; it is the absence of structured measurement before scale.

Matt Doughty
Span-level scoring to catch behavioral quality collapse in agents
Evaluation

Span-level scoring to catch behavioral quality collapse in agents

Infrastructure dashboards stay green while agents hallucinate, skip steps, or delete data. Span-level scoring and schema validation catch what uptime monitors

Matt Doughty
How to measure model routing cost and quality in real time
Evaluation

How to measure model routing cost and quality in real time

Routing agents to cheaper models saves money only if quality holds. Instrument spans at the routing decision point to validate both sides.

Matt Doughty
How to measure task allocation drift in AI agents
Evaluation

How to measure task allocation drift in AI agents

AI agents drift from assigned work without triggering alerts. Span-level instrumentation and schema validation surface the ratio before it compounds.

Matt Doughty
Measuring what agents actually cost: token metrics that matter
Evaluation

Measuring what agents actually cost: token metrics that matter

Vendor claims of 60, 90% token savings don't survive contact with production benchmarks. Here's how to measure what your agents actually cost.

Matt Doughty
How to structure agent spans for fast human review
Evaluation

How to structure agent spans for fast human review

Spans structured with goal context, downstream projections, and confidence basis let reviewers reach a decision in under thirty seconds.

Matt Doughty
AI agent benchmark gaps: what to measure in production
Evaluation

AI agent benchmark gaps: what to measure in production

Enterprise AI agents show a 37% gap between benchmark scores and production performance. Trace-based evaluation and multi-dimensional scoring close it.

Matt Doughty
Audit trails for AI agents: what the architecture requires
Evaluation

Audit trails for AI agents: what the architecture requires

Append-only fact stores and provenance tracking let teams reconstruct every agent decision, but only if the design supports it from day one.

Matt Doughty
How to validate agent behavior against declared scope
Evaluation

How to validate agent behavior against declared scope

A behavior schema checked at each execution span catches file deletions and hallucinated packages before they reach production. Logs alone cannot.

Matt Doughty
OpenAI Hugging Face agent incident: what evaluation missed
Evaluation

OpenAI Hugging Face agent incident: what evaluation missed

Two OpenAI models ran 17,000 actions across a weekend and reached Hugging Face infrastructure. Logs recorded it; nothing evaluated it in time.

Matt Doughty
Why generated docs beat static documentation for AI agents
Developer Experience

Why generated docs beat static documentation for AI agents

Static docs rot and agents treat them as fact. Generate reference docs from code at read time; hand-write only the decisions the code cannot explain.

Josh Gillies
MCP vs API: what is the difference?
MCP

MCP vs API: what is the difference?

APIs let any two programs communicate. MCP is a standard that lets AI agents discover and call APIs the same way everywhere, without custom wiring for each on

Matt Doughty
Agentic AI vs generative AI: what is the difference?
Comparisons

Agentic AI vs generative AI: what is the difference?

Generative AI produces content from a prompt. Agentic AI wraps a generative model in a goal-seeking loop that plans, calls tools, and acts.

Matt Doughty
LangChain vs LangGraph: what is the difference?
Comparisons

LangChain vs LangGraph: what is the difference?

LangChain provides components for building LLM apps. LangGraph adds stateful, graph-based orchestration for agents. Most complex agents use both.

Matt Doughty
MCP vs A2A: What's the Difference?
MCP

MCP vs A2A: What's the Difference?

MCP connects an agent to tools and data. A2A connects agents to each other. Two complementary protocols for agent systems, and where each one fits.

Matt Doughty
RAG vs fine-tuning: which one does your use case need?
Comparisons

RAG vs fine-tuning: which one does your use case need?

RAG adds external knowledge at query time. Fine-tuning bakes behaviour into the weights. Most teams eventually need both, for different reasons.

Matt Doughty
RAG vs MCP: What's the Difference?
MCP

RAG vs MCP: What's the Difference?

RAG gives a model knowledge from documents; MCP gives an agent tools and live systems. What each does, how they differ, and why most agents use both.

Matt Doughty
Why AI agents accumulate step-level errors after passing benchmarks
Evaluation

Why AI agents accumulate step-level errors after passing benchmarks

Agents that score well in testing still produce compounding step-level errors in live workflows. Those errors grow invisible until the cost is significant.

Matt Doughty
Does token overhead variance signal agent behavioral instability?
Evaluation

Does token overhead variance signal agent behavioral instability?

Token overhead variance reveals whether an agent follows the same internal path across invocations. Consistent overhead matters more than low overhead.

Matt Doughty
How to catch agent quality decay before it reaches users
Evaluation

How to catch agent quality decay before it reaches users

Intervention rate, scored task completion, and cross-agent error propagation signal agent decay days before user complaints appear.

Matt Doughty
Why agent tests pass but production fails
Evaluation

Why agent tests pass but production fails

Offline evaluations miss 30 to 40 percent of real-world failure modes. Layering trace-based and online scoring catches what test sets cannot.

Matt Doughty
Why production AI agents fail where benchmarks pass
Evaluation

Why production AI agents fail where benchmarks pass

Production agents fail 70 to 95 percent of the time depending on task complexity. Benchmarks test agents in isolation, not inside live workflows.

Matt Doughty
Why AI agent quality gaps appear after production scale
Evaluation

Why AI agent quality gaps appear after production scale

Most AI agent pilots pass internal evals but degrade sharply under production load. This article explains the three measurement gaps responsible.

Matt Doughty
How to measure token overhead in AI agents before scaling
Evaluation

How to measure token overhead in AI agents before scaling

Separating system, retrieval, and productive tokens reveals a 4x cost gap between identical agent outputs that total-session metrics hide.

Matt Doughty
AI agent observability vs evaluation: what each one does
MCP

AI agent observability vs evaluation: what each one does

Observability records what your agent did. Evaluation scores whether it was correct. You need both, and most teams only have one.

Matt Doughty
How to measure AI agent quality in production
Evaluation

How to measure AI agent quality in production

Task success rate, cost per task, drift, and span-level latency are the five metrics that reveal whether a production agent is working or quietly failing.

Matt Doughty
What are ghost actions in AI agents?
Evaluation

What are ghost actions in AI agents?

Ghost actions are unauthorized agent steps that task-level monitoring never records. They compound silently and can exhaust resources before anyone notices.

Matt Doughty
What Customers Ask Before They Trust Your AI Agent
Evaluation

What Customers Ask Before They Trust Your AI Agent

The five questions enterprise buyers ask before they'll rely on your AI agent, and the evidence artifacts that answer each one before a deal stalls.

Matt Doughty
5 Questions Every Head of AI Should Ask About Agent Governance
Product releases

5 Questions Every Head of AI Should Ask About Agent Governance

Scaling AI agents from pilots to production requires governance infrastructure. These five questions help Heads of AI evaluate whether their organisation can scale agents responsibly.

Matt Doughty
5 Questions Every ML Engineer Should Ask About Agent Runtime Controls
Product releases

5 Questions Every ML Engineer Should Ask About Agent Runtime Controls

ML engineers building AI agents need runtime controls that work with their development workflow — not against it. These five questions help evaluate agent governance from an engineering perspective.

Matt Doughty
5 Questions Every AI Product Manager Should Ask About Agent Governance
Product releases

5 Questions Every AI Product Manager Should Ask About Agent Governance

AI product managers must balance user experience with governance requirements. These five questions help PMs ship agent-powered products that are both useful and responsible.

Matt Doughty
5 Questions Every Risk Manager Should Ask About AI Agent Deployments
Product releases

5 Questions Every Risk Manager Should Ask About AI Agent Deployments

AI agents introduce risk categories that traditional risk frameworks do not cover. These five questions help risk managers evaluate and mitigate the unique risks of autonomous AI agents.

Matt Doughty
5 Questions Every CISO Should Ask Before Deploying AI Agents
Product releases

5 Questions Every CISO Should Ask Before Deploying AI Agents

AI agents introduce attack surfaces that traditional security tools were not designed for. These five questions help CISOs evaluate whether their organisation is ready to deploy agents safely.

Matt Doughty
5 Questions Every AI Governance Lead Should Ask About Agent Oversight
Product releases

5 Questions Every AI Governance Lead Should Ask About Agent Oversight

AI governance frameworks designed for models do not cover agents. These five questions help governance leads extend their programmes to address the unique challenges of autonomous AI agents.

Matt Doughty
Taming the Lobster: Announcing Prefactor’s Integration with OpenClaw
Product releases

Taming the Lobster: Announcing Prefactor’s Integration with OpenClaw

Prefactor announces a new integration with OpenClaw (Clawdbot)

Matt Doughty
How to Analyze Multi-Agent AI Attack Surfaces
Security

How to Analyze Multi-Agent AI Attack Surfaces

Framework to inventory agents, map dependencies, detect context poisoning and prompt injection, and apply behavioral and static analysis to secure multi-agent AI.

Matt Doughty
Data Retention for AI Agents in Regulated Industries
Compliance

Data Retention for AI Agents in Regulated Industries

Guidance on AI agent log retention across healthcare, finance, and EU/US law—recommended retention periods, privacy controls, and centralized compliance practices.

Matt Doughty
Claude x Canva Remote MCP server demo
Product releases

Claude x Canva Remote MCP server demo

Live demo showing how to connect Cursor to Canva using Remote MCP — enabling secure agent access across tools.

Matt Doughty
Prefactor x Claude Remote MCP server demo
Product releases

Prefactor x Claude Remote MCP server demo

Live demo showing how to connect Cursor to Canva using Remote Model Context Protocol (MCP) — enabling secure, agent-driven access between tools.

Matt Doughty
MCP server vs MCP client: what is the difference?
MCP

MCP server vs MCP client: what is the difference?

An MCP server exposes tools, resources and prompts; an MCP client is the connector inside an AI application that talks to one server. The two roles, how a request flows between them, and how to tell whether the agent behind the client is calling the right tools.

Matt Doughty
MCP vs LLM: what is the difference?
MCP

MCP vs LLM: what is the difference?

An LLM decides what to do next; MCP carries its tool calls to outside systems and brings the results back. How the two fit together, what goes wrong when they do, and how to evaluate an agent that uses MCP.

Matt Doughty

See how every agent performs, and make it better

Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.

Book a demoView docs