Prompt caching
Prompt caching stores the processed form of a stable prompt prefix so that repeated requests reusing it are billed at a steep discount and processed faster.
When requests share a long, unchanging beginning — a system prompt, a document, a tool schema — a provider can cache the work of processing that prefix and reuse it. Cached input tokens are charged at a fraction of the normal rate.
The catch is the word prefix: only a shared, stable start caches, so ordering prompts with fixed content first and variable content last maximizes the benefit. It is one of the few optimizations that cuts cost with no effect on output quality.
Newsletter
Learn this stuff by email
New definitions, tools, and explainers when I publish them. No spam, unsubscribe anytime.