# Model routing

> VI: https://5ac.vn/wiki/model-routing/ · EN: https://5ac.vn/en/wiki/model-routing/ · lang: en · spec_as_of 2026-10-19 (spec no formal spec) · cluster C8

The layer that decides which AI model serves each task — by cost, latency, context length and sensitivity — with the routing reason written to the log.

---
## Definition

Model routing is the layer that decides which model serves each request. Instead of one model carrying every job, a router reads the task — its cost, latency, context length, language and sensitivity — and hands it to the model that fits. A router is not a model; it is policy that runs in front of models. In 5ac's control-plane framing, model routing sits in the same row as access rights and logs: low-risk work goes to a cheap model, sensitive work to controlled infrastructure, and only quality-critical jobs earn a premium model.

## Mechanism

5ac describes a minimum four-layer architecture: a task classifier splits work into reading, drafting, analysis, tool calls, data checks and decisions needing human review; the model router selects by cost, latency, context length, language and business risk; a policy guardrail hides sensitive data, limits tool permissions and checks secrets; a cost ledger records tokens, runtime, which agent ran and what came out. A good router also records why it chose — so a decision is auditable six months later. Without routing, businesses overpay for simple work or under-provision important decisions. The research side: RouteLLM (arXiv:2406.18665) trains routers on human preference data, cutting cost by more than 2x on some benchmarks without dropping answer quality — and the router keeps its accuracy when you swap the strong/weak pair underneath it. UniRoute (arXiv:2502.08773) handles models the router never saw at training time, representing each model as a feature vector and sending each prompt to the smallest model that can answer it, tested across 30+ models.

## Application context

For Vietnamese businesses, routing is the first cost lever in a mix-model strategy: DeepSeek takes long-form text and Vietnamese context, Gemini Flash takes fast multimodal steps, premium models stand at the control points — contracts, legal risk, pricing calls, security review. The strategic prize is structural: when API prices, export rules or model quality shift, you change the router instead of rewriting every workflow, and the model layer can route to DeepSeek, OpenAI, Claude or a local model on cost and risk alone. The condition that comes with it: a router must run alongside permissions, audit logs and an emergency stop — the mix is only trustworthy when all four are in place.

## Sources fetched

- [DeepSeek and Gemini mix: cost optimization for agentic apps in Vietnamese SMBs](/en/blog/deepseek-gemini-mix-cost-optimization-agentic-apps-vietnamese-smbs/) — the four-layer architecture, router records its reason — accessed 2026-10-19
- [What Is an AI Agent Control Plane? Governance for Safe Enterprise Automation](/en/blog/ai-agent-control-plane-governance/) — routing by cost, sensitivity and difficulty — accessed 2026-10-19
- [On-prem multi-agent OS for Vietnamese SMBs: lower cost, safer data, faster Zalo integration](/en/blog/multi-agent-os-on-prem-vietnamese-businesses/) — change the router, not the workflows — accessed 2026-10-19
- <https://arxiv.org/abs/2406.18665> — RouteLLM: Learning to Route LLMs with Preference Data (v4) — verified via arXiv API 2026-10-19
- <https://arxiv.org/abs/2502.08773> — Universal Model Routing for Efficient LLM Inference (v2) — verified via arXiv API 2026-10-19

## Related

- [Token cost control](/en/wiki/ai-token-cost/)
- [Agent orchestration](/en/wiki/agent-orchestration/)
- [Vendor lock-in](/en/wiki/vendor-lock-in/)
- [Audit log](/en/wiki/audit-log/)
- [Inference engine](/en/wiki/inference-engine/)