Glossary term
What is prompt caching (Prompt (prefix) caching)?
prompt caching stands for Prompt (prefix) caching.
Reusing the work done for an unchanged start of a prompt, so repeat requests are cheaper and faster.
Learn it properly · AI EngineeringLesson 3: Cost, Latency and Caching for LLM FeaturesToken arithmetic, time to first token, prompt and response caching, model routing, batching, timeouts and budgets, with a simulation.