+ P Observe, evaluate, and improve your Pydantic AI agents
Capture every Pydantic AI tool call as the typed event the SDK already emits — FunctionToolCallEvent in, FunctionToolResultEvent out — plus the RunContext each call carries.
What Prefactor records from Pydantic AI
Pydantic AI + Prefactor
Observe for Pydantic AI
Prefactor observes your Pydantic AI agents in real time — every LLM call, tool invocation, and custom span captured as s
Open → EvaluateEvaluate for Pydantic AI
Prefactor evaluates your Pydantic AI agents — score outcome quality against the captured spans, track drift by comparing
Open → ObserveAct for Pydantic AI
Prefactor acts on your Pydantic AI agents at runtime — block, throttle, sandbox, or escalate a tool call or data access
Open →How the Pydantic AI integration works
- Tool calls surface as typed events — FunctionToolCallEvent and FunctionToolResultEvent — available via agent.iter() or run_stream_events(); Prefactor spans map onto these events directly rather than re-parsing raw model output.
- Every tool call carries a RunContext (parameterized by the agent's deps_type), and Pydantic AI's own instrumentation is already built on OpenTelemetry — Prefactor's spans sit at the same boundary, so nothing about the agent's dependency injection needs to change to observe it.
- Because Pydantic AI validates every output against a schema already, a failed validation is itself a real signal — not something Prefactor has to separately detect.
- Beyond auto-captured spans, use withSpan to record any custom step you define — an API call, a quality check, a business action.
Pydantic AI integration FAQ
Do I need a dedicated package for Pydantic AI?
You can instrument Pydantic AI today with the framework-agnostic prefactor-core SDK; a dedicated package can be added on request.
What does Prefactor capture from Pydantic AI?
Prefactor records agent runs, tool calls, output validation steps and LLM calls as structured, timestamped spans — so every Pydantic AI run is captured as trace data you can reconstruct, search and export end to end.
Does Prefactor add latency or change how Pydantic AI runs?
No. Observability capture is designed to stay off your agent's critical path, so it doesn't alter your Pydantic AI logic or your users' responses. The only part that acts inline is the optional runtime guardrails you enable per agent — by design, so a high-risk or low-confidence action can be held for human approval before it executes.
Can I evaluate agents built with Pydantic AI and catch regressions?
Yes. Once runs are captured, eval suites score quality and groundedness on real traffic, drift detection flags behaviour changes after deployment, and versioned eval history catches regressions before they ship — the observe → evaluate → improve loop applied to your Pydantic AI agents.
Related guides
See it on your Pydantic AI agents
Book a 15-minute setup and our team gets you tracing in production.