A model router takes a request and decides which model serves it, trading quality against cost and latency per call rather than per application. I got interested because I noticed I already run one: the routing table for my ilo development pipeline lives in my head, and it has hard-won entries. This post covers:
- The three places routing happens: gateway, application, and human
- What my hand-routing converged on, and why one entry got reverted
- What would convince me to automate it
Where routing lives
At the gateway layer, services like OpenRouter and the AI gateways front many providers behind one endpoint, so routing is a deployment concern: pick per request by price, latency, or availability, with fallback when a provider degrades. At the application layer, the code chooses a model per task type, a classifier or a config map deciding that summarisation goes cheap and legal review goes expensive. And below both sits the human layer, where someone just knows which model to use for what, which is where most real routing happens today.
My hand-built routing table
The pipeline I run for ilo development settled into fixed assignments: dogfooding persona agents run on Haiku, because the job is to use the language the way a cheap fast agent would and report friction, and the volume makes frontier pricing silly. Fix and feature subagents run on Sonnet. The interesting entry is that fixes ran on Opus for a stretch, and I reverted them to Sonnet: the quality delta on scoped, well-briefed fixes did not justify the cost delta once the brief included exact file paths and line numbers from an investigation phase.
That reversion is the whole routing problem in one anecdote. The right model depends less on the task’s difficulty than on how much context engineering surrounds it. A well-briefed cheap model beats a cold expensive one on scoped work, which means the router’s real input is not “how hard is this task” but “how good is the brief”, and no automated router I have seen takes that as a signal.
What would earn automation
Static assignments work because my task types are stable and few. A router earns its place when request types are heterogeneous and arrive unlabelled, which is exactly the chat-product case and exactly not the pipeline case. The failure I would watch for in either direction is silent tier drift: a router quietly downgrading a task that used to get the strong model, with nobody watching the quality regression. Routing decisions are cheap to change and expensive to notice, which argues for logging the model choice alongside every output, whatever layer makes the call.