Both trace agents, run experiments, score production traffic and version prompts. Datadog's case is correlation: agent traces next to the APM, infrastructure and RUM data you already send it; Opik's case is choice over where it runs, with a free Apache-2.0 self-hosted version. Whether you already pay Datadog usually decides this.[1][4]
The difference between Datadog and Opik is where each runs: Datadog Agent Observability, formerly LLM Observability, is a module of Datadog's hosted platform that needs a Datadog account, has no self-hosted option and ties agent traces to APM and infrastructure data. Opik, from Comet, is an Apache-2.0 platform you can self-host for free or use as a hosted cloud. Teams already on Datadog lean Datadog; teams that need self-hosting lean Opik.
| Decision factor | Datadog | Opik |
|---|---|---|
| Ownership | Datadog, a public company (Nasdaq: DDOG); needs a Datadog account | Comet ML, independent; reports $70M in total funding |
| Licence | Proprietary SaaS; instrumentation SDKs are open source | Apache-2.0 for the core platform and SDKs |
| Self-hosting | No; Datadog-hosted sites only | Free; all features except user administration |
| Blocking | Not in Agent Observability; needs AI Guard, a separate product in Limited Availability | Partial: guardrails your code calls; they fail closed |
| Code evals on live traffic | Run in your application, results submitted by API | Custom Python metrics run as online rules |
| Billing unit | LLM spans only; tool and agent spans not billed | All spans |
| Free tier | 40K LLM spans a month, 15-day retention | 25k spans a month, 60-day retention, up to 10 members |
| First paid tier | Pro: $160 a month billed annually for 100K LLM spans | Pro Cloud: $19 a month for 100k spans |
| Capability | Datadog | Opik | Prefactor |
|---|---|---|---|
| Observe | |||
| LLM call tracing | ✓ | ✓ | ✓ |
| Multi-step agent tracing (tool calls, nested spans) | ✓ | ✓ | ✓ |
| Cost tracking | ✓ | ✓ | ✓ |
| OpenTelemetry support | ✓ | ✓ | ✓ |
| Evaluate | |||
| Offline evals (datasets, experiments) | ✓ | ✓ | ✓ |
| Online evals on production traffic | ✓ | ✓ | ✓ |
| LLM-as-a-judge scoring | ✓ | ✓ | ✓ |
| Code-based evaluators | ✓ | ✓ | ✓ |
| Human review and annotation | ✓ | ✓ | ✓ |
| Prompt versioning and playground | ✓ | ✓ | — |
| Act on findings | |||
| Runtime controls on live agent actions | Partial | Partial | ✓ |
| Gateway or proxy in the request path | — | Partial | — |
| Licence and deployment | |||
| Open source | Partial | ✓ | Partial |
| Self-hosting | — | ✓ | Partial |
Datadog renamed LLM Observability to Agent Observability in June 2026. It is one product inside the Datadog platform and needs a Datadog account. Agent traces sit next to Datadog APM, infrastructure monitoring and RUM, so a slow agent step can be followed to the backend service behind it. It runs only on Datadog-hosted sites and is not available on Datadog's US government sites. Lapdog, a local trace viewer that works without an account, is for development rather than production.
Opik is a standalone product from Comet. Its core platform is Apache-2.0 and self-hosting is free, with every feature except user administration; production deployments run on Kubernetes, since the local Docker install is described as not production-ready. Opik Cloud is the hosted alternative. If your telemetry does not live in Datadog, or must stay on your own infrastructure, Opik fits where Datadog cannot.
Datadog runs experiments against versioned datasets, which can be built from production traces, with repeated runs, CI/CD automation and a browser Playground. Custom LLM-as-a-judge evaluators and Datadog's built-in evaluators run on targeted production spans with optional sampling, at span, trace or session level, using your own connected model provider account. Custom code evaluators run automatically only in experiments; in production you run them in your application and submit the results. Annotation Queues cover human review, and prompt versioning includes per-environment deployment and A/B Prompt Experimentation, with some parts in Preview.
Opik covers the same ground with more than 30 built-in metrics, custom judge prompts and an Agent as a Judge that reads traces with tools. Custom Python metrics run in experiments and also as online rules on production threads, so code checks on live traffic run inside Opik rather than in your application. Annotation queues, a Prompt Library with environment-tagged versions and a Prompt Optimizer with six algorithms complete it.
Agent Observability monitors and evaluates but does not block. Blocking prompts, responses and tool calls is the job of AI Guard, a separate Datadog product billed per token scanned and in Limited Availability since June 2026. Opik includes guardrails in the same product: PII, topic, prompt injection, judge and custom classifier checks that run as a blocking call your code makes and handles. Opik's PII and topic checks are English only, and neither vendor documents approval holds for agent actions. In practice, a Datadog customer who wants blocking adds a second product and a second bill; an Opik user adds calls in application code.
Both accept OpenTelemetry without their own SDK. Datadog reads GenAI semantic conventions v1.37 or later and OpenInference, and recognises OpenLLMetry instrumentation, with an ingestion delay of 3 to 5 minutes. Opik accepts OTLP over HTTP only and does not state which conventions it reads. On native SDKs, Datadog's Java SDK auto-instruments only OpenAI and its Go SDK is experimental; Opik ships SDKs for Python, TypeScript, Java and .NET.
Datadog bills only LLM spans, meaning calls to a model provider; tool, workflow, agent, retrieval and embedding spans are not billed. As of 20 October 2026, Free covers 40K LLM spans a month with 15-day retention, and Pro is $160 a month billed annually ($200 month-to-month) for the first 100K LLM spans, then $3.50 per 10K on annual terms. Retention beyond 15 days is a paid add-on, priced per 10K spans for 30, 60 or 90 days.
Opik counts every span. Free Cloud covers 25k spans a month with 60-day retention, and Pro Cloud is $19 a month for 100k spans, then $5 per 100k. An agent that makes many tool calls for each model call will cost less on Datadog's unit than its raw span count suggests, so compare on your own traces using Datadog's pricing and Opik's pricing.
We sell the layer this section describes. Read it with that in mind.
Stated plainly, since Prefactor publishes this page: Prefactor is an agent evaluation platform that scores each step of a run and can act on a failing score through runtime policies, approval holds and a kill switch for a run, an agent or a team. Datadog offers blocking through AI Guard, a separate product, and Opik through guardrails your code calls; neither documents approval holds. Prefactor has no prompt playground or prompt registry and its platform is hosted, and it does not correlate with APM data the way Datadog does. See the full comparison of 15 platforms.
Reviewed against public product and documentation pages on October 20, 2026. If a vendor has changed a feature, product name, or positioning since then, send a correction and we will update it. Numbered source links in the page body point to the ordered sources below.
Prefactor scores each step of every agent run and can hold or stop an action when a score says it should not happen. Start on the free tier: 5,000 spans a month, no credit card required.
Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.