Both are general observability vendors with agent tracing, evals and prompt versioning. Datadog is the natural choice if your services already report to Datadog; Logfire suits teams that want one OpenTelemetry platform, a self-hosted path on Enterprise and a gateway in the same product. The APM vendor you already pay for usually settles this.[1][4]
Datadog Agent Observability (formerly LLM Observability) and Logfire both put agent traces beside the rest of your application telemetry; the difference is that Datadog's is a module of the Datadog SaaS platform with no self-hosted option, while Logfire is a standalone OpenTelemetry platform from Pydantic that can be self-hosted on Enterprise. Datadog bills only LLM spans, and Logfire bills every record. For blocking, Datadog sells AI Guard separately, and Logfire's AI gateway covers model calls.
| Decision factor | Datadog | Logfire |
|---|---|---|
| Ownership | Datadog, public company (Nasdaq: DDOG) | Pydantic, independent |
| Requires | A Datadog account; one module of the Datadog platform | Standalone; any OpenTelemetry SDK can send data |
| Self-hosting | None; Datadog-hosted only | Enterprise, on Kubernetes via Helm |
| What is billed | LLM spans only; tool and agent spans are not billed | All records: spans, logs and metrics |
| Run-time blocking | AI Guard, a separate product in limited availability | Gateway guardrails and spend caps on model calls |
| SDKs | Python, Node.js, Java, Go (experimental), HTTP API | Python, JavaScript/TypeScript, Rust |
| Free tier | 40K LLM spans a month, 15-day retention | Personal: 10M records a month, 1 seat, 30-day retention |
| First paid tier | Pro: $160 a month billed annually, 100K LLM spans | Team: $49 a month, 10M records, 5 seats |
| Capability | Datadog | Logfire | Prefactor |
|---|---|---|---|
| Observe | |||
| LLM call tracing | ✓ | ✓ | ✓ |
| Multi-step agent tracing (tool calls, nested spans) | ✓ | ✓ | ✓ |
| Cost tracking | ✓ | ✓ | ✓ |
| OpenTelemetry support | ✓ | ✓ | ✓ |
| Evaluate | |||
| Offline evals (datasets, experiments) | ✓ | ✓ | ✓ |
| Online evals on production traffic | ✓ | ✓ | ✓ |
| LLM-as-a-judge scoring | ✓ | ✓ | ✓ |
| Code-based evaluators | ✓ | ✓ | ✓ |
| Human review and annotation | ✓ | Partial | ✓ |
| Prompt versioning and playground | ✓ | ✓ | — |
| Act on findings | |||
| Runtime controls on live agent actions | Partial | Partial | ✓ |
| Gateway or proxy in the request path | — | ✓ | — |
| Licence and deployment | |||
| Open source | Partial | Partial | Partial |
| Self-hosting | — | Partial | Partial |
Datadog renamed LLM Observability to Agent Observability in June 2026; the documentation still sits under the old path. It is one product inside the Datadog platform and needs a Datadog account, and its value comes from correlating agent traces with the APM and infrastructure data already there. It accepts OTLP traces directly without the Datadog SDK or Agent, using OpenTelemetry GenAI conventions v1.37 or later or OpenInference, with a three to five minute ingestion delay. It is not available on Datadog's US government sites.
Logfire is a standalone platform. Its SDKs are OpenTelemetry wrappers that record HTTP requests, database queries, logs and metrics together with model and tool calls, and any standard OpenTelemetry SDK can send data. For a team without an APM vendor, Logfire can cover both jobs. For a team that already runs Datadog, adding Logfire means a second place to look during an incident, which is often the deciding cost.
SDK coverage is uneven on the Datadog side. Python and Node.js are well covered, the Java SDK has one auto-instrumented integration (OpenAI) and the Go SDK is experimental with manual instrumentation only; any language can use the HTTP API or OTLP. Logfire's richest tooling is also Python-first, with JavaScript, TypeScript and Rust SDKs alongside. For local work, Datadog's Lapdog shows traces on your machine without a Datadog account.
The two are closer here than their positioning suggests. Datadog runs experiments against versioned datasets, which can be built from production traces, plus a browser Playground. Custom LLM judges run automatically on targeted production spans with sampling, using your own connected provider account. Code evaluators run automatically only in experiments; in production you run them in your application and submit results through the SDK or HTTP API. Datadog lists DeepEval, Pydantic Evals and NVIDIA NeMo evaluators as supported, so pydantic-evals code can feed either platform.
Logfire's live evaluations also run in your application, in the background at a sample rate, and report results as OpenTelemetry events. For people, Datadog has annotation queues; Logfire's run annotations are in beta and queues are early access. Both version prompts, fetch them at runtime and control rollout with labels or targeting rules. Datadog adds A/B tests of prompt versions, with some sub-features in Preview; Logfire adds a Prompt Playground that replays prompts from captured traces.
Agent Observability itself monitors and evaluates but does not block. Blocking of prompts, responses and tool calls is AI Guard, a separate Datadog product billed per token scanned and listed as Limited Availability in June 2026. It is the only option of the two that names tool calls in scope. Logfire's gateway, part of Logfire since March 2026, applies per-rule observe, flag, redact or block actions to model calls and enforces hard spend caps, but it does not cover tool actions and neither vendor documents approval holds.
As of 20 October 2026, Datadog's Free plan covers 40K LLM spans a month with 15-day retention. Pro is $160 a month billed annually ($200 month to month, $240 on demand) for the first 100K LLM spans, then $3.50 per 10K on annual terms. Retention beyond 15 days is an add-on, from $6.50 per 10K spans for 30 days. Only calls to an LLM provider are billed; tool, workflow, agent, retrieval and embedding spans are not.
Logfire bills every record: Personal is free for 10M records and one seat, Team is $49 a month with 10M records and five seats, Growth is $249 a month with unlimited seats, and extra records cost $2 per million. An agent with many tool calls per model call meters lightly on Datadog; an application with many non-LLM spans meters heavily on Logfire. Compare Datadog's pricing and Logfire's pricing on your own traffic.
We sell the layer this section describes. Read it with that in mind.
Stated plainly, since Prefactor publishes this page: Prefactor is an agent evaluation platform that scores each step of a run and acts on what it finds through runtime policies on tool calls and data access, approval holds for risky actions and a kill switch for a run, an agent or a team. Datadog's AI Guard, sold separately and in limited availability, can block tool calls; Logfire's gateway covers model calls. Neither documents approval holds. Prefactor is not an APM tool, has no prompt playground or prompt registry, and its platform is hosted. See the full comparison of 15 platforms.
Reviewed against public product and documentation pages on October 20, 2026. If a vendor has changed a feature, product name, or positioning since then, send a correction and we will update it. Numbered source links in the page body point to the ordered sources below.
Prefactor scores each step of every agent run and can hold or stop an action when a score says it should not happen. Start on the free tier: 5,000 spans a month, no credit card required.
Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.