+ P Observe, evaluate, and improve your LlamaIndex agents
Capture every LlamaIndex Workflow step the same way LlamaIndex itself defines one — event in, work done, event out — so a trace reads as the same step graph you wrote.
What Prefactor records from LlamaIndex
LlamaIndex + Prefactor
Observe for LlamaIndex
Prefactor observes your LlamaIndex agents in real time — every LLM call, tool invocation, and custom span captured as st
Open → EvaluateEvaluate for LlamaIndex
Prefactor evaluates your LlamaIndex agents — score outcome quality against the captured spans, track drift by comparing
Open → ObserveAct for LlamaIndex
Prefactor acts on your LlamaIndex agents at runtime — block, throttle, sandbox, or escalate a tool call or data access b
Open →How the LlamaIndex integration works
- A LlamaIndex Workflow is event-driven: a @step method receives a typed Event, does its work, and returns another Event that triggers the next step — Prefactor spans one step at a time, the same StartEvent-to-StopEvent path the workflow itself runs.
- Retrieval steps built on index.as_query_engine() are captured with the same span shape as any other step, so a RAG-style retrieve-then-synthesize workflow traces as one connected run, not a retrieval log plus a separate LLM log.
- Beyond auto-captured spans, use withSpan to record any custom step you define — an API call, a quality check, a business action.
LlamaIndex integration FAQ
Do I need a dedicated package for LlamaIndex?
You can instrument LlamaIndex today with the framework-agnostic prefactor-core SDK; a dedicated package can be added on request.
What does Prefactor capture from LlamaIndex?
Prefactor records query-engine steps, retrievals, agent tool calls and LLM calls as structured, timestamped spans — so every LlamaIndex run is captured as trace data you can reconstruct, search and export end to end.
Does Prefactor add latency or change how LlamaIndex runs?
No. Observability capture is designed to stay off your agent's critical path, so it doesn't alter your LlamaIndex logic or your users' responses. The only part that acts inline is the optional runtime guardrails you enable per agent — by design, so a high-risk or low-confidence action can be held for human approval before it executes.
Can I evaluate agents built with LlamaIndex and catch regressions?
Yes. Once runs are captured, eval suites score quality and groundedness on real traffic, drift detection flags behaviour changes after deployment, and versioned eval history catches regressions before they ship — the observe → evaluate → improve loop applied to your LlamaIndex agents.
Related guides
See it on your LlamaIndex agents
Book a 15-minute setup and our team gets you tracing in production.