Glossary term
What is model routing? Meaning for developers
Sending easy requests to a cheap model and escalating hard ones to a stronger model.
Learn it properly · AI EngineeringLesson 3: Cost, Latency and Caching for LLM FeaturesToken arithmetic, time to first token, prompt and response caching, model routing, batching, timeouts and budgets, with a simulation.