Prompt caching
Also known as: caching · prefix caching
Prompt caching reuses the expensive computation (the KV cache) for a prefix of the prompt that repeats across requests — like a shared system prompt or a large fixed document.
The first call pays full price; matching later calls are much cheaper and faster.
- System prompts — cached and reused constantly.
- Large reference docs — same prefix, many questions.
The money trick: if your app repeats a big preamble, caching is one of the cheapest performance and cost wins available. Put the stable stuff at the front.