Long context
Also known as: long context window
Long context refers to models that accept very large prompts — from 100K to 1M+ tokens — so you can feed whole books, codebases, or conversation histories at once.
It’s impressive, with real caveats:
- Memory: the KV cache scales with length (see KV cache).
- Cost: you pay for every input token.
- Quality: context rot means long ≠ well-used.
Use it wisely: long context is a powerful hammer. But for "find the relevant part of a big corpus", RAG is usually cheaper and sharper. Know which tool the job needs.