Softmax
Also known as: soft-max
Softmax converts a set of raw scores (logits) into probabilities that add up to 1, so they can be sampled from.
The key property: it amplifies differences — the largest score gets a much larger share of the probability mass.
- Higher temperature → softmax is “softer” (flatter, more even).
- Lower temperature → softmax is “sharper” (more concentrated on the top).
Connect the dots: temperature doesn't act on probabilities directly — it acts on the *logits before softmax*. That's why it's such a clean way to control diversity.