Both trace agents, run experiments, score production traffic with LLM judges and offer annotation queues. Galileo, now Splunk Agent Observability, adds guardrails that act during a run and suits Splunk estates; Confident AI is the self-serve, evaluation-first option with an open-source metric library. How you buy software often decides this before features do.[1][6]
Confident AI is an independent evaluation platform built on the open-source DeepEval framework, with self-serve plans; Galileo has been owned by Cisco since May 2026 and sold as Splunk Agent Observability since 7 August 2026. Galileo can block harmful model and tool inputs and outputs during a run through Agent Control; Confident AI cannot. Confident AI offers a free tier and documented prompt version control, which Galileo does not list.
| Decision factor | Confident AI | Galileo |
|---|---|---|
| Ownership | Independent; Y Combinator Winter 2025 company | Cisco (completed May 2026); sold as Splunk Agent Observability since 7 August 2026 |
| Runtime guardrails | None; threshold checks run on recorded data | Yes: Agent Control checks model and tool inputs and outputs during a run |
| Open source | DeepEval and confident-trace (Apache-2.0) | SDKs and Agent Control (Apache-2.0) |
| Code evaluators | Code-Eval metrics in a platform sandbox | Yes, but not in the Splunk-hosted edition |
| Prompt versioning | Prompt Studio with commits, versions and labels | Prompts and playground; version control not documented |
| Self-hosting | Enterprise only, on your AWS, GCP or Azure Kubernetes | On-premises, including air-gapped (commercial) |
| Free tier | 2 seats, 1 GB-month of trace data, 5 test runs a week | None listed for Agent Observability |
| Paid pricing | Starter: $200 a month per organisation | From $100 a month per 1.2 million spans a year, billed annually |
| Capability | Confident AI | Galileo | Prefactor |
|---|---|---|---|
| Observe | |||
| LLM call tracing | ✓ | ✓ | ✓ |
| Multi-step agent tracing (tool calls, nested spans) | ✓ | ✓ | ✓ |
| Cost tracking | ✓ | ✓ | ✓ |
| OpenTelemetry support | ✓ | ✓ | ✓ |
| Evaluate | |||
| Offline evals (datasets, experiments) | ✓ | ✓ | ✓ |
| Online evals on production traffic | ✓ | ✓ | ✓ |
| LLM-as-a-judge scoring | ✓ | ✓ | ✓ |
| Code-based evaluators | ✓ | ✓ | ✓ |
| Human review and annotation | ✓ | ✓ | ✓ |
| Prompt versioning and playground | ✓ | Partial | — |
| Act on findings | |||
| Runtime controls on live agent actions | — | ✓ | ✓ |
| Gateway or proxy in the request path | — | — | — |
| Licence and deployment | |||
| Open source | Partial | Partial | Partial |
| Self-hosting | Partial | ✓ | Partial |
Cisco completed its acquisition of Galileo in May 2026, and since 7 August 2026 the product has been sold as Splunk Agent Observability inside Splunk Observability Cloud. Customers onboarded before that date use Galileo's documentation; later customers use Splunk's. The old pricing page redirects to Splunk, which lists annual billing and no free tier for this product.
Confident AI is a small independent company, listed by Y Combinator as a Winter 2025 company, with self-serve Free, Starter and Team plans in US or EU regions. The buying motion is the clearest difference: a free self-serve start on one side, an annual Splunk contract on the other. If you already run Splunk, keeping agent evaluation beside your other telemetry is a strong reason to stay there.
Galileo runs as SaaS inside Splunk Observability Cloud, or on-premises under a commercial licence, including air-gapped sites. Customers who joined before 7 August 2026 remain on the legacy Galileo cloud and documentation, so check which edition a quote refers to. Agent Control can also be self-hosted on its own from its open-source repository. Confident AI hosts in the US or EU on every plan, offers custom data residency on request, and self-hosts on Enterprise on AWS, GCP or Azure Kubernetes.
Galileo's Agent Control evaluates model and tool inputs and outputs while an agent runs and can block prompt injection, personal data leaks and harmful content using rules set in one place, without changes to agent code. It replaced the older Protect product in June 2026, and its core was released as open source under Apache-2.0 in March 2026. Blocking content is documented; approval workflows for actions are not. The hosted Agent Control is documented as an enterprise feature.
Confident AI documents no inline blocking. Its threshold checks assess recorded production data over a trailing 24-hour window and report pass or fail, and red-teaming detections flag traces after ingestion. If you need checks on what an agent sends to and receives from its tools while it runs, Galileo covers that and Confident AI does not.
Existing Protect users need to move to Agent Control, so a Galileo purchase today means adopting the newer layer; guides written before June 2026 describe Protect, which is now deprecated.
Confident AI is stronger on the development side. DeepEval runs more than 40 metrics locally and in CI, Code-Eval runs custom Python metrics in a sandbox on the platform, Prompt Studio versions prompts with commits and environment labels, and red teaming and multi-turn and voice simulation test agents before release. Its online evals run only referenceless metrics during tracing.
Galileo offers experiments, multi-turn evaluation, Autotune for judge metrics, and Luna-2 small models as lower-cost evaluators. Check one limit carefully: the Splunk-hosted edition supports five preset Luna evaluators and no custom code evaluators or Luna fine-tuning, which are available on-premises only. Its prompts page does not document version control.
Both have structured human review. Galileo's Annotation Queues became generally available in August 2026, alongside annotations on sessions, traces and spans. Confident AI's annotation queues cover traces, spans, threads and, since October 2026, test cases, with multi-reviewer forms and an end-user feedback API.
As of 20 October 2026, Splunk lists Agent Observability from $100 a month per 1.2 million spans a year, billed annually, including 500 million Luna tokens and five Luna fine-tuning runs a year. No free tier is listed. Confident AI's Free plan covers 2 seats and 1 GB-month of trace data, Starter is $200 a month per organisation with unlimited seats and 5 GB-months, Team is $2,000 a month, and extra storage is $1 per GB-month.
Galileo prices by span count on an annual commitment; Confident AI by stored data, priced monthly. See Confident AI's pricing and Splunk's pricing for Agent Observability.
We sell the layer this section describes. Read it with that in mind.
Stated plainly, since Prefactor publishes this page: Prefactor is an agent evaluation platform that scores each step of a run and acts on what it finds through runtime policies on tool calls and data access, approval holds for risky actions, and a kill switch for a run, an agent or a team. Of these two, Galileo's Agent Control is the closest to that, though it blocks content rather than holding actions for approval; Confident AI checks recorded data after the fact. Prefactor is self-serve with a free tier and is not tied to Splunk, but it has no prompt playground or prompt registry and its platform is hosted. See the full comparison of 15 platforms.
Reviewed against public product and documentation pages on October 20, 2026. If a vendor has changed a feature, product name, or positioning since then, send a correction and we will update it. Numbered source links in the page body point to the ordered sources below.
Prefactor scores each step of every agent run and can hold or stop an action when a score says it should not happen. Start on the free tier: 5,000 spans a month, no credit card required.
Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.