Both trace agents, run experiments on datasets and score production traffic, and both accept OpenTelemetry. Arize lets you start free on your own infrastructure with Phoenix; Datadog puts agent traces next to the APM and infrastructure data you already send it. Since Dynatrace bought Arize, each now belongs to a competing observability suite.[3][4]
Arize gives you a free tool to run yourself (Phoenix) and a hosted platform (AX), while Datadog Agent Observability, formerly LLM Observability, is a hosted module of the Datadog platform with no self-hosted option. Dynatrace completed its acquisition of Arize on 1 October 2026, and Phoenix and AX continue. Both read OpenTelemetry and OpenInference traces, so the wider observability suite you already run usually decides it.
| Decision factor | Arize | Datadog |
|---|---|---|
| Ownership | Acquired by Dynatrace, completed 1 October 2026; Phoenix and AX continue | Datadog, a public company (Nasdaq: DDOG); needs a Datadog account |
| Products | Phoenix (self-hosted) and Arize AX (hosted, or self-hosted on Enterprise) | Agent Observability, renamed from LLM Observability in June 2026 |
| Self-hosting | Phoenix free; AX on Enterprise, including air-gapped | None; Lapdog is a local development viewer only |
| Licence | Phoenix: Elastic License 2.0 (source-available); AX: commercial | Proprietary SaaS; SDKs such as dd-trace-py are open source |
| OpenTelemetry | Built on it; Arize maintains the OpenInference conventions | Accepts OTLP with GenAI conventions v1.37+ or OpenInference; 3 to 5 minute ingestion delay |
| Run-time blocking | Partial: AX guards run in your own code via Guardrails AI | Partial: needs AI Guard, a separate product in Limited Availability |
| Free tier | AX Free: 25k spans a month, 15-day retention; Phoenix free to self-host | Free: 40K LLM spans a month, 15-day retention |
| First paid tier | AX Pro: $50 a month, 50k spans, 30-day retention | Pro: $160 a month billed annually, 100K LLM spans |
| Capability | Arize | Datadog | Prefactor |
|---|---|---|---|
| Observe | |||
| LLM call tracing | ✓ | ✓ | ✓ |
| Multi-step agent tracing (tool calls, nested spans) | ✓ | ✓ | ✓ |
| Cost tracking | ✓ | ✓ | ✓ |
| OpenTelemetry support | ✓ | ✓ | ✓ |
| Evaluate | |||
| Offline evals (datasets, experiments) | ✓ | ✓ | ✓ |
| Online evals on production traffic | ✓ | ✓ | ✓ |
| LLM-as-a-judge scoring | ✓ | ✓ | ✓ |
| Code-based evaluators | ✓ | ✓ | ✓ |
| Human review and annotation | ✓ | ✓ | ✓ |
| Prompt versioning and playground | ✓ | ✓ | — |
| Act on findings | |||
| Runtime controls on live agent actions | Partial | Partial | ✓ |
| Gateway or proxy in the request path | Partial | — | — |
| Licence and deployment | |||
| Open source | Partial | Partial | Partial |
| Self-hosting | ✓ | — | Partial |
Since 1 October 2026 this has been a choice between two observability suites. Dynatrace completed its acquisition of Arize that day and says Phoenix and AX will continue, with Arize capabilities integrated into Dynatrace over time. Datadog Agent Observability, renamed from LLM Observability in June 2026, is already one module of the Datadog platform and needs a Datadog account.
If your services already report to Datadog, its module links an agent's traces to APM, infrastructure and RUM data for the same request, and the cost lands on a bill you already pay. If you run Dynatrace, Arize is the product that will move closer to it. If you run neither, expect Arize's packaging to shift as the integration proceeds, and check the terms before a large rollout.
Arize has a free route: Phoenix installs with pip, Docker or Kubernetes and includes tracing, datasets, experiments, LLM and code evaluators and prompt versioning. Its licence is the Elastic License 2.0, which lets you modify and self-host the code but not offer it to others as a hosted service. Arize AX can also run on your own Kubernetes cluster, connected, semi-restricted or air-gapped, on the Enterprise plan.
Datadog documents no self-hosted deployment. Lapdog runs a local trace viewer without a Datadog account, but it is a development tool, not a production option. Agent Observability is also not available on Datadog's US government sites. If traces must stay inside your network, Arize is the only one of the two that allows it.
One limit on the free route: continuous evaluation of production traffic, monitors and alerting are AX features, and Phoenix's own documentation points to AX for them. Self-hosted Phoenix covers development and offline evaluation, not scoring live traffic.
Both score live traffic with sampling. AX runs evaluator tasks at span, trace or session level, with LLM judges, code evaluators and, since September 2026, agent-as-a-judge on every plan. Datadog runs custom LLM-as-a-judge and built-in evaluators on targeted production spans, at trace or session level as well, using your own provider account for the judge model.
Code evaluators work differently. In Datadog they run automatically only inside Experiments; in production you run them in your application and submit the results through the SDK or HTTP API. Arize AX lists code evaluators alongside LLM judges for online evals, and Phoenix added Python and TypeScript code evaluators in Docker sandboxes in October 2026.
For multi-agent systems, AX draws an agent trajectory graph, set automatically for LangGraph, CrewAI and AG2. Datadog's agent monitoring shows tools used and handoffs for OpenAI Agents SDK, LangGraph and CrewAI in Python, and auto-instruments Claude Agent SDK, Google ADK, Pydantic AI and Strands.
As of 20 October 2026, both offer a free hosted tier with 15-day retention: AX Free covers 25k spans a month and Datadog's Free plan covers 40K LLM spans a month. AX Pro is $50 a month for 50k spans with 30-day retention. Datadog Pro is $160 a month billed annually, or $200 month to month, for the first 100K LLM spans, then $3.50 per 10K, with retention beyond 15 days sold as an add-on.
Both AX Free and AX Pro include unlimited users, evals and experiments. AX Enterprise is custom priced and is the only plan that self-hosts AX; Phoenix stays free to self-host.
The units differ. Datadog bills only spans that call an LLM provider; tool, agent, workflow, retrieval and embedding spans are not billed. An agent that makes several tool calls for each model call therefore counts for less on Datadog's meter than its total span count suggests. Run a week of your own traces against Arize's pricing and Datadog's pricing before you commit.
We sell the layer this section describes. Read it with that in mind.
Stated plainly, since Prefactor publishes this page: Prefactor is an agent evaluation platform that scores each step of a run and can act on what it finds through runtime policies on tool calls and data access, approval holds for risky actions, and a kill switch for a run, an agent or a team. Neither product here does that from its evaluation layer: Arize AX documents guards that run in your own code around model calls, and Datadog sells blocking separately as AI Guard, in limited availability. Prefactor has no prompt playground or prompt registry and its platform is hosted, so if free self-hosting or prompt iteration is the requirement, Phoenix or Datadog's prompt tooling is the better fit. See the full comparison of 15 platforms.
Reviewed against public product and documentation pages on October 20, 2026. If a vendor has changed a feature, product name, or positioning since then, send a correction and we will update it. Numbered source links in the page body point to the ordered sources below.
Prefactor scores each step of every agent run and can hold or stop an action when a score says it should not happen. Start on the free tier: 5,000 spans a month, no credit card required.
Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.