Freezing (weights)
Also known as: weight freezing · frozen layers
Freezing means locking some of a model’s weights so the training process can’t update them. The rest adapts; the frozen part stays exactly as trained.
It’s central to PEFT/LoRA (the base is frozen) and a common trick to prevent catastrophic forgetting.
- Freeze all, train adapters → LoRA-style.
- Freeze early layers, train later → keep low-level features, adapt high-level.
The benefit: freezing is how you *specialize* a model without *corrupting* it. It's the safety belt of fine-tuning.