Case StudyCrypto / Blockchain infrastructure

Support agents that know when to stop guessing

A conversation that stalls on a screenshot or an ambiguous step is a real-time judgement call, not a static FAQ.

Prefactor is an agent evaluation platform for teams building AI agents: it observes every agent run from development to production, evaluates whether the agent did its job, and shows how to improve it.

Built on LangGraph — a real design-partner deployment, anonymised.

case-file · Crypto / Blockchain infrastructurelive
LangGraph logoAgent runs on LangGraph
Triggers handoff
Resolution scored
LLM-as-judge checks the handoff actually resolved it; thumbs up/down attaches to the same conversation
  • Real-time conversation monitoring, not after-the-fact log review
  • Human-in-the-loop handoff built into the span structure
  • One conversation, one inspectable instance
TL;DR

How do you stop a support agent guessing when it is out of its depth?

Watch each conversation live and hand off at the point it stalls. In this LangGraph deployment, Prefactor treats every conversation as its own instance and marks with custom spans exactly where the agent stalled, such as on a screenshot, an ambiguous step or a user who is clearly stuck. At that point it can trigger a human handoff instead of letting the agent guess. Personal data is tagged and redacted as it appears, and each resolution is scored.

The problem

A support agent walks users through a long setup flow, connecting an account, verifying a step, completing a multi-stage process. Some users get stuck partway through and send a screenshot instead of describing the problem in words. The agent can attempt to interpret it, but interpretation isn't certainty, and guessing wrong at a high-stakes step is worse than asking a human. There's a quality dimension underneath this too: the same question can get a different answer every time, and fixing one thing tends to quietly break another, which is exactly why a one-off benchmark score doesn't hold up, and why the real ask is closer to a SWE-bench-style pass/fail suite plus an LLM judge scoring the qualitative stuff a binary check can't.

LangGraph logo
AgentBuilt on LangGraph
Conversation runs
User describes the issue
User gets stuck
Sends a screenshot instead of a clear answer
Human takes over
Resolves the specific step
Triggers handoff
Prefactor logo
PrefactorWatching every step, in real time
Every step captured as a span
Real time, not logged after the fact
Pattern detected
Recognizes the stuck signal as it happens
Resolution scored
LLM-as-judge checks the handoff actually resolved it; thumbs up/down attaches to the same conversation

How it works in Prefactor

Real-time conversation monitoring reads the conversation as it unfolds, watching for the pattern that signals a user is stuck rather than just asking a normal question, not logging what happened after the fact.

Human-in-the-loop is a first-class handoff, built directly into the span structure, not a support ticket thrown over a wall. When the stuck pattern is detected, Prefactor triggers the handoff as a designed part of the flow.

Every support conversation is its own inspectable instance, open exactly one interaction and see everything that happened in it, rather than digging through a shared log.

Custom spans capture qualitative detail: which step of the flow the user got stuck on, whether the handoff was timely, tagged as structured data on the conversation, not buried in a transcript.

Quality gets scored on the same conversation, not just risk: an LLM-as-judge pass checks whether the handoff actually resolved the issue, and a thumbs up/down from the human who took over attaches directly to that instance, so "did this actually work" has an answer, not just "was this risky."

Sensitive data in a screenshot or message gets tagged the moment it's encountered, and can be redacted automatically.

If something goes wrong, that specific run can be paused, a checkpoint, not a termination, or killed outright, triggered natively from the dashboard or programmatically via a custom span, without touching any other conversation the agent is handling.

Frequently asked questions

Does Prefactor replace our support agent's logic?
No: it sits alongside it, watching the conversation and making the handoff decision. Your agent keeps doing what it does; Prefactor decides when a human should step in.
What counts as a "stuck" pattern worth escalating?
That's configurable, repeated clarifying questions, a screenshot with no accompanying explanation, a user repeating the same step. You define what should trigger a handoff for your flow.
Can we see why a handoff happened after the fact?
Yes: the conversation instance keeps full context: what led up to the handoff, what was tagged, and what happened once a human took over.

See it on your own agents

Book a demo and we'll walk through support agents that know when to stop guessing on a fleet like yours — real frameworks, real traces.

Agent Performance Platform
Unified performance platform for agents, authentication, and risk management
All Systems Operational
3Global Agents
7Instances
5Services
12%Human Intervene
4High Risk
$2,360Monthly Spend
Mission ControlLive agent health with 7-day activity heartbeat
Claims Proc...68
$330/moRed
Claims Proc...65
$160/moRed
Claims Proc...82
$170/moAmber
ChatGPT74
$150/moAmber

See how every agent performs, and make it better

Prefactor helps teams observe, evaluate, and improve their AI agents in production, across every framework and provider.