Top-p (nucleus sampling)
Also known as: nucleus sampling · nucleus
Top-p (nucleus sampling) keeps the smallest set of tokens whose cumulative probability reaches p (e.g., 0.9), then samples only from that set.
It adapts to the shape of the distribution:
- Confident prediction → tiny nucleus (near-deterministic).
- Many plausible options → larger nucleus (more variety).
Typical values: 0.8–1.0.
Why people prefer it: unlike top-k (a fixed count), top-p is *distribution-aware*. It's usually the single sampling knob to tune first.