QLoRA
Also known as: QLoRA
QLoRA combines quantization with LoRA: the base model is quantized (compressed) and frozen, while a small LoRA adapter is trained on top.
It made fine-tuning large models dramatically more accessible:
- Base model stored at low precision (4-bit).
- Only the LoRA weights trained at full precision.
- Result: fine-tune a 65B+ model on a single high-end GPU.
Why it mattered: QLoRA is a big reason fine-tuning went from "research lab only" to "a good GPU can do it". It collapsed the memory wall.