AI 101

PEFT

Also known as: parameter-efficient fine-tuning

Training

PEFT (Parameter-Efficient Fine-Tuning) is a family of techniques that adapt a large model by training only a small fraction of its parameters — leaving the bulk of the weights frozen.

The headline method is LoRA. The payoff is big:

  • Less memory — you don’t need to hold gradients for every weight.
  • Faster, cheaper — smaller update set.
  • Swappable adapters — one base model, many thin LoRA “skill” files.

This is how you fine-tune locally: PEFT is what makes fine-tuning a 7B–70B model feasible on a single GPU instead of a datacenter.