Dropout
Also known as: dropout regularization
Dropout is a regularization technique: during training, a random fraction of units are switched off on each pass, so the network can’t depend on any particular one.
It forces redundant, robust representations — a classic defense against overfitting.
- During training: random units off each forward pass.
- During inference: all units on (scaled appropriately).
The intuition: it's like forcing a team to play with random members out, so no single player is indispensable. The team gets more robust.