AI triage for support: routing, not deflecting

The support automation industry has a success metric, and it's the wrong one. "Deflection" — the share of customers kept away from human agents — is the number on every vendor deck, and it produces exactly what it measures: systems optimised to hold the door shut. Customers feel it. They spend four minutes fencing with a bot that paraphrases their question back, learn that typing "agent" repeatedly works, and arrive at a human angrier than they started. The company hits its deflection target and loses the customer.

Good AI triage has the opposite objective: route people to the right outcome as fast as possible, where "the right outcome" is often an automated answer and often a person who's already read the story. Routing is an information-design problem with a model attached. Deflection is a cost-containment fantasy with a chat box attached. This piece is the routing playbook — illustrated throughout by the constraints we hit building the intake assistant in the Beacon Health triage case study, where routing wrongly wasn't an annoyance, it was a clinical safety issue.

Design the intents before the bot

Triage quality is a taxonomy problem wearing a model costume. Before any prompting, sit with the support team and enumerate intents from real tickets — not the top ten, the whole long tail, and specifically the categories where a wrong automated answer is expensive: refunds in flight, account compromise, medical or legal questions, anything a customer in distress touches. At a typical SaaS or commerce company you'll end up with 25–60 intents.

Give every intent three fields in the routing table:

• Automatable? Fully automated answer, human-with-draft, or straight-to-human.
• Severity. Routine, urgent (billing lockouts, security), safety-critical.
• Handoff destination. Which queue, which SLA.

The list of straight-to-human intents — usually 15–25% of the total — is the design. A triage system that has never said "this one goes to a person immediately" has not been designed; it has been deflected onto a classifier. Safety-critical intents get the strongest rule: a single high-confidence match routes to a human with no bot conversation at all, because engagement is not the goal — correct treatment is.

Err toward humans on the margin

Classification has a dial, and the industry pushes it in the wrong direction. We set explicit precision/recall targets per intent: for straight-to-human categories, tune for recall — better to err toward routing a routine billing question to a person than to let an account-takeover attempt chat with the FAQ bot. For automatable intents, tune for precision — only auto-answer when confidence is genuinely high. The multi-class "confidence" score the model emits is roughly decorative in its raw form; we calibrate it against the golden set, per intent, before trusting a threshold. This is the same measurement posture as evals before features, applied to routing instead of generation.

The margin behaviours matter as much as the thresholds. Medium confidence gets a clarifying question — not a bot answer. "Just to make sure I've got this right: is the payment missing from your account, or were you charged twice?" Clarifiers are free, honest, and dramatically better than a confidently wrong answer; they also make the customer feel heard, which is most of what support is for.

The handoff is the product

Here's the part almost nobody builds: when the bot hands off, it transfers a package, not a transcript. A human agent receiving "Customer says billing issue" with a chat log attached has received nothing useful. The handoff package we design includes:

• Detected intent + confidence — "double charge, 0.9, confirmed by clarifier."
• Structured facts — account ID, order numbers, dates, amounts, already elicited from the customer.
• What's been tried — bot steps completed, docs surfaced, so the human never repeats a step. (Repeating a step is the #1 agent-friction complaint in every support workshop we've run.)
• Customer state — sentiment signal, how long they've been in the flow, messages sent.

The agent's first screen is a summary card, not a chat log. They walk in pre-briefed, like a nurse taking over a ward round. Handle time after a packaged handoff is typically 30–45% shorter than cold assignment — and the customer isn't asked to re-tell the story, which halved repeat-contact rates in one engagement we ran. The queue design patterns from human-in-the-loop review queues apply directly here: the model drafts, the human decides, the interface does the admin.

The line that builds trust

At the moment of handoff, show the customer one specific sentence: "You're talking to a person now — Sam from support. She's seen your order number and the notes so far." Name, context, proof the baton passed. It's a small line of copy doing heavy work: it separates the bot era of the conversation from the human era, it signals that repeating yourself is over, and it gives permission to be slightly less courteous if they've been fencing with the bot. We A/B tested three variants of this line on a retail client; the version naming the agent and citing a specific fact lifted post-handoff CSAT by nine points over the silent handoff.

This is a rare case where disclosure and experience align. Customers don't resent a bot that did the intake admin well; they resent discovering the bot only after asking a nuanced question. Earlier honesty ("I'll check what I can, and bring in a person if needed") plus the handoff line is the trust-maximising combination. It's the same principle behind streaming-thought interfaces from streaming UX: show the working before the answer.

Kill the deflection metric; measure escalation quality

If the team's dashboard is deflection rate, you'll build a wall. Replace it with metrics that can't be gamed by keeping people out:

• Resolved-in-bot CSAT — if automated answers aren't rated as well as human ones, your bot is deflecting, not resolving.
• Escalation rate by intent, over time — climbing escalations in an intent means the bot is drifting or the product changed.
• Repeat-contact-within-72h — the true resolution signal. Deflected customers who email again tomorrow are a cost, not a win.
• Agent time on packaged handoffs vs cold — proves or kills the handoff investment.
• Percentage of straight-to-human intents caught before bot contact — the safety metric. If a safety intent ever gets three turns of bot, that conversation is a postmortem.

The broader instrumentation posture — logging conversations as reviewable events, annotating failures — follows the same pattern as analytics for AI features: measure the movement, not the moment.

What we refuse to build

A short list, offered as a service: bots that hide the "speak to a person" path; sentiment analysis used to deprioritise unhappy customers; auto-close timers that punish slow typers; intent models trained on the support team's wishful taxonomy instead of real tickets. Any of these in a brief from our AI products practice triggers a design review before a line of code — not because we're precious, but because every one of them is a measurable loyalty leak wearing a cost saving.

Routing is the humble, high-yield version of support AI. It doesn't demo as well as a fully generative concierge. It just works, treats people like adults, and frees the human team for the conversations where humans are the point.

Key takeaways

• Design intent taxonomies from real tickets, and mark 15–25% of intents straight-to-human with bypass routing.
• Calibrate confidence per intent; err toward humans for safety categories, toward clarifiers in the grey zone.
• Hand off a package — intent, facts, steps tried, customer state — never a bare transcript.
• Announce the human with one specific sentence naming the agent and a fact they've seen.
• Replace deflection rate with resolved-in-bot CSAT, repeat-contact rate and escalation quality; deflection rewards walls.

FAQ

Won't routing-first produce worse cost-per-ticket than aggressive deflection? In the first quarter, sometimes. By month six, no — because deflection strategies generate repeat contacts, escalated churn and social media screenshots. Routing-first reduces total contacts and lifts CSAT, which compounds in ways the ticket-cost line never shows.

How big should the clarification loop be before we just hand off? One clarifier, maximum two. If the intent is still unresolved after that, the model doesn't understand the customer and the kindest act is a human. Bots that keep asking follow-ups feel like interrogation.

Do we need a different system for chat vs email? The triage and routing layer is shared; only the conversation surface differs. Email intake benefits even more from packaged handoffs because the agent receives the package before composing a single reply.

When is full automation appropriate for an intent? When the answer is deterministic and verifiable: order status, password reset, invoice re-send. If the right response depends on judgment or the cost of a wrong answer is high, it belongs in human-with-draft or straight-to-human — regardless of what the model can technically do.