Learning rate
Also known as: lr
The learning rate sets how large each weight update is. It’s the single most sensitive knob in training.
- Too high: the model overshoots, oscillates, and can diverge (loss explodes).
- Too low: it learns glacially slowly and may stall in a bad spot.
Good training is largely the art of finding the right rate (and often scheduling it down over time).
Analogy: it's the step size for someone walking down a slope. Big steps get you there fast — or send you flying off the cliff.