Both pair a free tool with a commercial platform, and both put continuous production evals in the commercial part. Arize's free tool, Phoenix, is a full self-hosted app with tracing and a UI; Confident AI's free tool, DeepEval, is a code-first metrics framework with no UI of its own. Which free half you want usually decides it.[3][4]
Arize gives you a complete self-hosted app for free (Phoenix), while Confident AI gives you a free evaluation framework (DeepEval) and keeps its app hosted unless you buy Enterprise. Dynatrace completed its acquisition of Arize on 1 October 2026 and says Phoenix and AX continue; Confident AI is an independent Y Combinator company. In both cases, continuous evaluation of production traffic lives in the commercial platform.
| Decision factor | Arize | Confident AI |
|---|---|---|
| Ownership | Acquired by Dynatrace, completed 1 October 2026; Phoenix and AX continue | Independent; Y Combinator Winter 2025 company |
| Free tool | Phoenix: self-hosted app with tracing, datasets, experiments and evaluators | DeepEval: open-source metrics framework; no UI or persistence on its own |
| Licence of the free tool | Elastic License 2.0 (source-available) | Apache-2.0 (DeepEval and confident-trace) |
| Online evals | Arize AX only | Confident AI platform only; referenceless metrics during tracing |
| Self-hosting the commercial platform | AX Enterprise, including air-gapped | Enterprise only, licence key and private images |
| Runtime guards | Partial: AX guards run in your code via Guardrails AI | None; threshold checks run on recorded data |
| Free hosted tier | AX Free: 25k spans a month, 15-day retention, unlimited users | Free: 2 seats, 1 GB-month of trace data, 5 test runs a week |
| First paid tier | AX Pro: $50 a month, 50k spans | Starter: $200 a month per organisation, unlimited seats |
| Capability | Arize | Confident AI | Prefactor |
|---|---|---|---|
| Observe | |||
| LLM call tracing | ✓ | ✓ | ✓ |
| Multi-step agent tracing (tool calls, nested spans) | ✓ | ✓ | ✓ |
| Cost tracking | ✓ | ✓ | ✓ |
| OpenTelemetry support | ✓ | ✓ | ✓ |
| Evaluate | |||
| Offline evals (datasets, experiments) | ✓ | ✓ | ✓ |
| Online evals on production traffic | ✓ | ✓ | ✓ |
| LLM-as-a-judge scoring | ✓ | ✓ | ✓ |
| Code-based evaluators | ✓ | ✓ | ✓ |
| Human review and annotation | ✓ | ✓ | ✓ |
| Prompt versioning and playground | ✓ | ✓ | — |
| Act on findings | |||
| Runtime controls on live agent actions | Partial | — | ✓ |
| Gateway or proxy in the request path | Partial | — | — |
| Licence and deployment | |||
| Open source | Partial | Partial | Partial |
| Self-hosting | ✓ | Partial | Partial |
Dynatrace completed its acquisition of Arize on 1 October 2026. Its announcement says Arize will keep supporting both Phoenix and AX and that Arize capabilities will be integrated into Dynatrace over time. For a team already on Dynatrace that is a reason to look closer; for everyone else it means packaging and pricing may move in the next few quarters.
Confident AI is independent. The Y Combinator directory lists it as a Winter 2025 company in San Francisco with a small team. It ships weekly: October 2026 alone added tool failure analytics, span trees for subagent branches, a prompt-based evaluator and voice connections for simulating conversations. A small vendor moving quickly is worth weighing against a large acquirer that has promised continuity.
This is the clearest split between the two. Phoenix is an application: you install it with pip, Docker or Helm and get a UI with traces, datasets, experiments, LLM and code evaluators, annotations, a playground and prompt versioning. You can run your whole development loop on it without paying anyone. Its licence, the Elastic License 2.0, lets you modify and self-host the code but not offer it to others as a hosted service.
DeepEval is a library. It runs test cases locally and in CI with more than 40 metrics, including agent, multi-turn, RAG and MCP metrics, and it is Apache-2.0. On its own it has no UI and stores nothing; results, datasets, regression reports and traces live in the hosted Confident AI platform. The confident-trace SDK, released in September 2026, is also Apache-2.0 and can export to your own OpenTelemetry collector instead.
So a team that wants a free dashboard on its own servers will find Phoenix covers more. A team that wants evaluation as code in its test suite, with a permissive licence, will find DeepEval fits better and can add the hosted platform later.
Continuous scoring of live traffic is a paid-platform feature for both. Arize AX runs evaluators as tasks against production spans, traces or sessions with sampling and filters, and Phoenix's documentation points to AX for this. Confident AI runs evaluation tasks on traces, spans and threads as they arrive, with filters and sample rates, but only referenceless metrics run during tracing.
On acting during a run, Arize goes slightly further. AX documents guards built on the Guardrails AI framework that run in your own code around model calls and can return a default response or re-ask. Confident AI documents no inline blocking: its threshold checks assess recorded production data over a trailing 24-hour window and report pass or fail. Neither documents approval steps or policies on an agent's tool calls.
Human review and cost tracking are close. Phoenix lets you annotate traces through its UI, client or REST API, and AX adds annotation queues for spans, traces, sessions and dataset examples. Confident AI has annotation queues for traces, spans, threads and test cases, multi-reviewer forms, an end-user feedback API and alignment between metric and human scores. On cost, Phoenix calculates spend from token counts with a built-in pricing table and custom prices; Confident AI infers cost automatically only for OpenAI, Anthropic and Gemini models, so other providers need rates you set.
They price on different units. As of 20 October 2026, Arize AX prices by spans: AX Free is $0 for 25k spans a month with 15-day retention and unlimited users, and AX Pro is $50 a month for 50k spans with 30-day retention. Enterprise is custom and is the only way to self-host AX. Phoenix is free to self-host.
Confident AI prices per organisation, not per seat, with trace storage in GB-months: Free covers 2 seats, 1 project and 1 GB-month of spans (excess is dropped), Starter is $200 a month with unlimited seats and 5 GB-months, and Team is $2,000 a month with 75 GB-months and SSO. Extra storage is $1 per GB-month. Check Arize's pricing and Confident AI's pricing against your own span counts and payload sizes.
We sell the layer this section describes. Read it with that in mind.
Stated plainly, since Prefactor publishes this page: Prefactor is an agent evaluation platform that scores each step of a run and can act on a failing score through runtime policies on tool calls and data access, approval holds for risky actions, and a kill switch for a run, an agent or a team. Arize AX's guards run in your code around model calls, and Confident AI checks recorded data after the fact; neither holds a tool call for approval. Prefactor has no prompt playground or prompt registry and its platform is hosted, so if a free self-hosted app or an open-source metrics library is what you need, Phoenix or DeepEval is the better fit. See the full comparison of 15 platforms.
Reviewed against public product and documentation pages on October 20, 2026. If a vendor has changed a feature, product name, or positioning since then, send a correction and we will update it. Numbered source links in the page body point to the ordered sources below.
Prefactor scores each step of every agent run and can hold or stop an action when a score says it should not happen. Start on the free tier: 5,000 spans a month, no credit card required.
Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.