Routing: the request sent to the wrong specialist

A support system has a router in front of three specialist agents: billing, shipping and technical. Each specialist has its own tools and its own prompt. The router is one model call: "Classify the message as billing, shipping or technical. Reply with one word." A customer wrote: "The app says my order failed but my card was charged $149. Now I can't log in either." The router said technical. The technical agent sent a password reset link and told the customer the order would show correctly once they were back in. Status success.

The charge was never looked at. The customer is $149 down, has a reset link they did not ask for, and will write again. The run is seven lines.

Line 3 is where it shows. The message had two problems in it, a charge with no order and a login failure, and the router was only allowed one word. Three labels, no "unsure", no "both". So it picked. Routers tend to pick on the last sentence or the most concrete noun, and "can't log in" is concrete. The failure was built into the prompt at line 1: a classifier that cannot say "I do not know" will always say something.

Then the specialist made it worse. The technical agent had reset_password and check_service_status. It had no tool for charges and no instruction to hand back a message it could not handle. So it did what it could, reset the password, and wrote a sentence about the order that it had no way to check. A wrong route plus a specialist that never refuses is the common shape of this bug. Each half looks fine on its own.

Nobody saw it because nothing measured it. There was no label on the ticket saying what the right route was, so there was no routing accuracy number, so a misroute looked like a resolved ticket.

Give the router a way out. The output is a small JSON object, not a word: route, confidence, and a second route when the message has two problems. Add an "unsure" value. Below a confidence you set from data, the ticket goes to a person, or to a general agent that can ask one clarifying question. Anthropic's 2024 guide on building effective agents describes routing this way: classify, then send to a specialized follow-up, and it works where the categories are distinct and the classification can be done accurately. A route is a decision with a failure mode, and the failure mode needs somewhere to go.

Make the handoff carry context and allow a return. In the OpenAI Agents SDK a handoff is a tool call that passes the conversation to another agent; the specialist should also have a tool to pass it back with a reason. The technical agent in this run should have said: this message has a charge in it and I have no tool for charges, route to billing. One sentence in its prompt, one extra tool.

Measure it. Label 200 real tickets by hand with the right route. Run the router over them every time the prompt or the model changes. Track two numbers in production: the misroute rate on the sample you spot check each week, and the share of tickets re-opened within seven days, because a re-opened ticket is often a misrouted one. Routing also decides cost. The RouteLLM paper (2024) showed that sending easy requests to a cheap model and hard ones to a strong one cut cost by more than half in some benchmarks while keeping quality close to the strong model alone. The design is the same: a confidence, a threshold, and a labelled set to tune it.

Line first. "Line 3. The router picked technical for a message that was about a charge and a login, because it was only allowed one word and had no unsure option. I would make it return a route and a confidence, add unsure, send low-confidence tickets to a person, and let the specialist hand back. Then I would label a few hundred tickets so the misroute rate is a number." If they ask what you would measure, name the two numbers and say why the second one matters: re-opens are where misroutes hide.