Glossary term
What is token? Meaning for developers
A small piece of text a model reads or writes. Cost and delay follow token counts.
Learn it properly · AI EngineeringLesson 3: Cost, Latency and Caching for LLM FeaturesToken arithmetic, time to first token, prompt and response caching, model routing, batching, timeouts and budgets, with a simulation.