+ P Act for LlamaIndex agents
Capture every LlamaIndex Workflow step the same way LlamaIndex itself defines one — event in, work done, event out — so a trace reads as the same step graph you wrote.
Prefactor acts on your LlamaIndex agents at runtime — block, throttle, sandbox, or escalate a tool call or data access before it executes, stop an agent instantly with the kill switch, and delete or redact tagged PII the moment it's encountered.
How to add act to LlamaIndex
Same SDK, no extra package
The prefactor-core client you already initialized for tracing is what enforcement runs through.
Wrap the action in a span
withSpan() wraps the operation you want governed — a tool call, a data write, anything your own code defines.
Policy runs before your code does
Prefactor evaluates policy inside withSpan before calling the function you wrapped — block, throttle, or hold for approval, before the action executes.
# pip install prefactor-core
import os
from prefactor_core import PrefactorCoreClient, PrefactorCoreConfig
from prefactor_http import HttpClientConfig
config = PrefactorCoreConfig(
http_config=HttpClientConfig(
api_url="https://app.prefactorai.com",
api_token=os.environ["PREFACTOR_API_TOKEN"],
)
)
client = PrefactorCoreClient(config)
await client.initialize()
# then instrument your LlamaIndex run with spans — see docs.prefactor.aiShown with the Prefactor SDK — a first-class, working integration today.
How the LlamaIndex integration actually works
- A LlamaIndex Workflow is event-driven: a @step method receives a typed Event, does its work, and returns another Event that triggers the next step — Prefactor spans one step at a time, the same StartEvent-to-StopEvent path the workflow itself runs.
- Retrieval steps built on index.as_query_engine() are captured with the same span shape as any other step, so a RAG-style retrieve-then-synthesize workflow traces as one connected run, not a retrieval log plus a separate LLM log.
- Beyond auto-captured spans, use withSpan to record any custom step you define — an API call, a quality check, a business action.
What Prefactor captures for LlamaIndex agents
Runtime policy enforcement
Every tool call, API request, or data access is evaluated against your rules at the point the agent tries to act — block, throttle, sandbox, or escalate, in sub-millisecond time.
Kill switch
Stop a single run, an agent, or every agent a team owns — immediately, no code deploy, triggered natively from Prefactor or programmatically from a custom span.
Human-in-the-loop approval
A flagged action pauses execution and routes to the right approver with full context — what the agent was trying to do, why, and its recent history — through Slack, Teams, or email.
PII deletion
Find every field carrying a tag and remove it in one action, wherever it appears — plus real-time redact, block, or escalate the moment PII is first encountered.
Trace anything — not just LLM calls
The SDK captures LLM, tool, and agent spans automatically. With withSpan you wrap any operation in your own span type — an API call, a database query, a quality check, a business action — each with its own payload and schema. It all flows into the same Observe, Evaluate, and cost views.
import { withSpan } from "@prefactor/core";
// Wrap ANY operation in a span you define — an API call, a quality
// check, a business action — with its own spanType, inputs and schema
await withSpan(
{
name: "research competitor",
spanType: "research_competitor",
inputs: { competitor },
},
async () => {
const results = await search(competitor); // your tool / API calls
return summarize(results); // captured as one span
},
);Nest spans to capture business-level actions, and start with permissive schemas you tighten over time. Instrumentation strategy →
An example run, span by span
Illustrative — a single LlamaIndex run as nested spans.
Illustrative example.
Manual logging vs DIY vs Prefactor
| Capability | Manual | DIY OpenTelemetry | Prefactor |
|---|---|---|---|
| LLM / tool / agent spans | Hand-rolled | ✓ build it | ✓ via the SDK |
| Token usage captured per call | — | Build it | ✓ |
| Configurable capture & sampling | — | Partial | ✓ |
| Hosted Admin UI (agents, instances) | — | — | ✓ |
| Risk profiles & audit trail | — | — | ✓ |
LlamaIndex act — FAQ
How do runtime policies work for LlamaIndex agents?+
A policy engine sits at the point a LlamaIndex agent calls a tool, queries an API, or touches data — the same interception point Prefactor uses across every framework. Every attempted action is evaluated against the agent's identity, permissions, and current context before it's allowed to proceed.
Does enforcement add latency to LlamaIndex runs?+
No meaningfully — policy evaluation runs in sub-millisecond time, so enforcement doesn't add noticeable latency to the agent's response.
Can I kill a single LlamaIndex run without stopping the whole agent?+
Yes — the kill switch scopes to a single run, a single agent, or every agent a team owns. Trigger it from the Prefactor dashboard directly, or programmatically via a custom span the moment your own code detects a problem.
How does human-in-the-loop approval work for LlamaIndex?+
When a policy flags a LlamaIndex agent's action for escalation, execution pauses and an approval request goes out with full context. The approver's decision is logged and either lets the action proceed or cancels it.
How is PII actually deleted from LlamaIndex agent data?+
Detection and tagging happen continuously; deletion is one action against the tag — find every field carrying it for a given person or record, and remove it, wherever it appears across your LlamaIndex runs.
Do I need a dedicated package for LlamaIndex?+
You can instrument LlamaIndex today with the framework-agnostic prefactor-core SDK; a dedicated package can be added on request.
What does Prefactor capture from LlamaIndex?+
Prefactor records query-engine steps, retrievals, agent tool calls and LLM calls as structured, timestamped spans — so every LlamaIndex run is captured as trace data you can reconstruct, search and export end to end.
Does Prefactor add latency or change how LlamaIndex runs?+
No. Observability capture is designed to stay off your agent's critical path, so it doesn't alter your LlamaIndex logic or your users' responses. The only part that acts inline is the optional runtime guardrails you enable per agent — by design, so a high-risk or low-confidence action can be held for human approval before it executes.
Can I evaluate agents built with LlamaIndex and catch regressions?+
Yes. Once runs are captured, eval suites score quality and groundedness on real traffic, drift detection flags behaviour changes after deployment, and versioned eval history catches regressions before they ship — the observe → evaluate → improve loop applied to your LlamaIndex agents.
Keep going
Act for other frameworks
LangChain →CrewAI →LangGraph →OpenAI Agents SDK →Claude Agent SDK →Microsoft AutoGen →See it on your LlamaIndex agents
Book a 15-minute setup and our team gets you act in production.