+ P Act for Haystack agents
Capture every Haystack Pipeline as the directed graph it actually is — each Component's run as its own span, connected the same way Haystack connects component outputs to inputs.
Prefactor acts on your Haystack agents at runtime — block, throttle, sandbox, or escalate a tool call or data access before it executes, stop an agent instantly with the kill switch, and delete or redact tagged PII the moment it's encountered.
How to add act to Haystack
Same SDK, no extra package
The prefactor-core client you already initialized for tracing is what enforcement runs through.
Wrap the action in a span
withSpan() wraps the operation you want governed — a tool call, a data write, anything your own code defines.
Policy runs before your code does
Prefactor evaluates policy inside withSpan before calling the function you wrapped — block, throttle, or hold for approval, before the action executes.
# pip install prefactor-core
import os
from prefactor_core import PrefactorCoreClient, PrefactorCoreConfig
from prefactor_http import HttpClientConfig
config = PrefactorCoreConfig(
http_config=HttpClientConfig(
api_url="https://app.prefactorai.com",
api_token=os.environ["PREFACTOR_API_TOKEN"],
)
)
client = PrefactorCoreClient(config)
await client.initialize()
# then instrument your Haystack run with spans — see docs.prefactor.aiShown with the Prefactor SDK — a first-class, working integration today.
How the Haystack integration actually works
- A Haystack Pipeline is a directed multigraph of Components, each connected by which output feeds which input — Prefactor's spans follow that same graph, so a trace reads as the pipeline's real execution order, not a flat list of calls.
- Haystack's own tracing captures component input/output for debugging, disabled by default to avoid sending sensitive data to a backend; Prefactor's span capture is controlled separately via PREFACTOR_CAPTURE_INPUTS/_OUTPUTS, so the two don't need to be configured in lockstep.
- Beyond auto-captured spans, use withSpan to record any custom step you define — an API call, a quality check, a business action.
What Prefactor captures for Haystack agents
Runtime policy enforcement
Every tool call, API request, or data access is evaluated against your rules at the point the agent tries to act — block, throttle, sandbox, or escalate, in sub-millisecond time.
Kill switch
Stop a single run, an agent, or every agent a team owns — immediately, no code deploy, triggered natively from Prefactor or programmatically from a custom span.
Human-in-the-loop approval
A flagged action pauses execution and routes to the right approver with full context — what the agent was trying to do, why, and its recent history — through Slack, Teams, or email.
PII deletion
Find every field carrying a tag and remove it in one action, wherever it appears — plus real-time redact, block, or escalate the moment PII is first encountered.
Trace anything — not just LLM calls
The SDK captures LLM, tool, and agent spans automatically. With withSpan you wrap any operation in your own span type — an API call, a database query, a quality check, a business action — each with its own payload and schema. It all flows into the same Observe, Evaluate, and cost views.
import { withSpan } from "@prefactor/core";
// Wrap ANY operation in a span you define — an API call, a quality
// check, a business action — with its own spanType, inputs and schema
await withSpan(
{
name: "research competitor",
spanType: "research_competitor",
inputs: { competitor },
},
async () => {
const results = await search(competitor); // your tool / API calls
return summarize(results); // captured as one span
},
);Nest spans to capture business-level actions, and start with permissive schemas you tighten over time. Instrumentation strategy →
An example run, span by span
Illustrative — a single Haystack run as nested spans.
Illustrative example.
Manual logging vs DIY vs Prefactor
| Capability | Manual | DIY OpenTelemetry | Prefactor |
|---|---|---|---|
| LLM / tool / agent spans | Hand-rolled | ✓ build it | ✓ via the SDK |
| Token usage captured per call | — | Build it | ✓ |
| Configurable capture & sampling | — | Partial | ✓ |
| Hosted Admin UI (agents, instances) | — | — | ✓ |
| Risk profiles & audit trail | — | — | ✓ |
Haystack act — FAQ
How do runtime policies work for Haystack agents?+
A policy engine sits at the point a Haystack agent calls a tool, queries an API, or touches data — the same interception point Prefactor uses across every framework. Every attempted action is evaluated against the agent's identity, permissions, and current context before it's allowed to proceed.
Does enforcement add latency to Haystack runs?+
No meaningfully — policy evaluation runs in sub-millisecond time, so enforcement doesn't add noticeable latency to the agent's response.
Can I kill a single Haystack run without stopping the whole agent?+
Yes — the kill switch scopes to a single run, a single agent, or every agent a team owns. Trigger it from the Prefactor dashboard directly, or programmatically via a custom span the moment your own code detects a problem.
How does human-in-the-loop approval work for Haystack?+
When a policy flags a Haystack agent's action for escalation, execution pauses and an approval request goes out with full context. The approver's decision is logged and either lets the action proceed or cancels it.
How is PII actually deleted from Haystack agent data?+
Detection and tagging happen continuously; deletion is one action against the tag — find every field carrying it for a given person or record, and remove it, wherever it appears across your Haystack runs.
Do I need a dedicated package for Haystack?+
You can instrument Haystack today with the framework-agnostic prefactor-core SDK; a dedicated package can be added on request.
What does Prefactor capture from Haystack?+
Prefactor records pipeline component runs, retrievals, agent tool calls and LLM calls as structured, timestamped spans — so every Haystack run is captured as trace data you can reconstruct, search and export end to end.
Does Prefactor add latency or change how Haystack runs?+
No. Observability capture is designed to stay off your agent's critical path, so it doesn't alter your Haystack logic or your users' responses. The only part that acts inline is the optional runtime guardrails you enable per agent — by design, so a high-risk or low-confidence action can be held for human approval before it executes.
Can I evaluate agents built with Haystack and catch regressions?+
Yes. Once runs are captured, eval suites score quality and groundedness on real traffic, drift detection flags behaviour changes after deployment, and versioned eval history catches regressions before they ship — the observe → evaluate → improve loop applied to your Haystack agents.
Keep going
Act for other frameworks
LangChain →CrewAI →LangGraph →OpenAI Agents SDK →Claude Agent SDK →Microsoft AutoGen →See it on your Haystack agents
Book a 15-minute setup and our team gets you act in production.