Support agents that know when to stop guessing

The problem

A support agent walks users through a long setup flow — connecting an account, verifying a step, completing a multi-stage process. Some users get stuck partway through and send a screenshot instead of describing the problem in words. The agent can attempt to interpret it, but interpretation isn't certainty — and guessing wrong at a high-stakes step is worse than asking a human. There's a quality dimension underneath this too: the same question can get a different answer every time, and fixing one thing tends to quietly break another — which is exactly why a one-off benchmark score doesn't hold up, and why the real ask is closer to a SWE-bench-style pass/fail suite plus an LLM judge scoring the qualitative stuff a binary check can't.

Agent Built on LangGraph

Conversation runs

User describes the issue

User gets stuck

Sends a screenshot instead of a clear answer

Human takes over

Resolves the specific step

Triggers handoff

Prefactor Watching every step, in real time

Every step captured as a span

Real time, not logged after the fact

Pattern detected

Recognizes the stuck signal as it happens

Resolution scored

LLM-as-judge checks the handoff actually resolved it; thumbs up/down attaches to the same conversation

How it works in Prefactor

Real-time conversation monitoring reads the conversation as it unfolds, watching for the pattern that signals a user is stuck rather than just asking a normal question — not logging what happened after the fact.

Human-in-the-loop is a first-class handoff, built directly into the span structure, not a support ticket thrown over a wall. When the stuck pattern is detected, Prefactor triggers the handoff as a designed part of the flow.

Every support conversation is its own inspectable instance — open exactly one interaction and see everything that happened in it, rather than digging through a shared log.

Custom spans capture qualitative detail: which step of the flow the user got stuck on, whether the handoff was timely — tagged as structured data on the conversation, not buried in a transcript.

Quality gets scored on the same conversation, not just risk: an LLM-as-judge pass checks whether the handoff actually resolved the issue, and a thumbs up/down from the human who took over attaches directly to that instance — so "did this actually work" has an answer, not just "was this risky."

Sensitive data in a screenshot or message gets tagged the moment it's encountered, and can be redacted automatically.

If something goes wrong, that specific run can be paused — a checkpoint, not a termination — or killed outright, triggered natively from the dashboard or programmatically via a custom span, without touching any other conversation the agent is handling.