TL;DR
An LLM (large language model) is the model that reads the conversation and decides what to do next, including whether to call a tool. MCP (Model Context Protocol) is an open standard for how an AI application connects that model to tools and data. The model makes the decision; MCP carries the request to the tool and the result back. Neither one checks whether the decision was right, which is the job of agent evaluation.
What is an LLM?
A large language model takes text in and produces text out. Current models such as GPT, Claude and Gemini also support tool calling: given descriptions of the tools available, the model can reply with a structured request to call one, with arguments. The model never runs the call itself. Application code runs it and passes the result back for the model's next step. On its own, an LLM has no connection to your systems.
What is MCP?
The Model Context Protocol is an open standard, first published by Anthropic in November 2024, for connecting AI applications to outside tools and data. An MCP server offers tools (functions the model can ask to run), resources (data the application can read into the model's context) and prompts (reusable templates). An MCP client, running inside the AI application, connects to one server and passes requests to it. MCP contains no model, so the same server works whichever LLM the application uses.
MCP vs LLM: the core difference
| LLM | MCP | |
|---|---|---|
| What it is | A model | A protocol |
| Its job | Reads the context, decides the next step, writes the response | Carries tool calls and data between the application and outside systems |
| Where it runs | A model provider's API or your own inference servers | MCP servers, on the same machine or remote |
| Without the other | Answers from its context; can still call tools through custom code | Has nothing deciding which tool to call |
| Standardised? | Each provider has its own API and tool-calling format | One open specification (current revision: 2026-07-28) |
How do an LLM and MCP work together?
User request -> AI application -> LLM chooses a tool -> MCP client -> MCP server -> your API or database -> result back to the LLM
A support agent asked "where is order 4471?" shows the split. The order system's MCP server lists a lookup_order tool. The model asks to call it with order_id: 4471. The server queries the order API and returns the status, and the model writes the reply from that status. Tool calling works without MCP too; MCP pays off once you have several tools or several AI applications, because each tool is wrapped once and every application can use it.
What goes wrong when an LLM uses MCP tools?
MCP reports whether a tool call ran. It does not report whether it was the right call. The failures that cost the most in production are the ones where every call succeeds:
- The model picks the wrong tool, such as
search_orderswhen it neededlookup_order, and builds an answer from the wrong data. - The model picks the right tool with the wrong arguments, such as last month's date range, and gets a plausible result.
- The model calls the same tool again and again without making progress, and the run ends with a half-finished task.
- A model or prompt change shifts which tools get called, and nothing errors.
In every case the protocol log shows successful calls and the user gets a confident answer. Because MCP makes it easy to swap the LLM behind the same tools, the last failure is the one teams hit most often: the model changes, the tools stay the same, and behaviour drifts without anyone touching the integration.
How Prefactor evaluates agents that use MCP
Prefactor is an agent evaluation platform: it records every agent run, scores whether the agent did its job, and shows where to improve it. For an agent that uses MCP, that works in three steps.
Observe every tool call. Real-time tracing records every model call and tool call as a span the moment it happens, with its inputs, outputs, duration and outcome, so each MCP tool call sits in the context of the run that made it. Instrumentation is a TypeScript or Python SDK, native for LangChain, Claude, Vercel AI, OpenClaw and LiveKit, or OpenTelemetry ingest.
Score each step, not just the final answer. Quality assessment scores runs step by step, so a wrong tool or a bad argument shows up at the step where it happened instead of being averaged into one number. Comparing the same steps across model versions, prompts and environments turns a model swap into a measurable difference rather than a support ticket.
Act before a bad call lands. Runtime policies check a tool call against your rules before it runs, and approval routing can hold a high-risk call for a person to approve.
The quickest way to see this on your own agent is the open-source evals: deterministic checks that catch wrong tools, silent loops and half-finished runs in real traces, with no judge model and no token cost. To run the full loop on your own agents, start on the free tier: 5,000 spans a month at October 2026 pricing, no credit card required.