Top-k sampling
Also known as: top k
Top-k sampling keeps only the k most probable next tokens and samples from just those.
- k = 1: always the most likely token (greedy, most deterministic).
- k = 50: more variety, some risk of a weird pick.
- Large k: approaches the full distribution.
It’s simple but blind to the shape of the probabilities — a confident distribution and a flat one both get the same k.
When to use it: top-k is fine and fast, but top-p usually behaves better. Start with top-p; drop to top-k only if you need a hard cap.