Decoding
Also known as: decoding strategy
Decoding is the overall strategy for turning a model’s per-token probabilities into a full output sequence. It’s the layer where “what could it say” becomes “what it does say.”
Main families:
- Greedy — always the top token.
- Beam search — keep the top few paths.
- Sampling — draw from the (shaped) distribution.
The lever you control: picking a decoding strategy + setting (temperature/top-p) is the primary way to change a model's behavior *without* changing the model at all.