Model merging
Also known as: merging · weight averaging
Model merging combines the weights of two or more models into a single new model — without retraining from scratch.
Simple techniques (averaging weights, task arithmetic) can surprisingly well blend capabilities:
- Merge a coding fine-tune + a writing fine-tune → a model that does both.
- Cheaper than training a new model from scratch.
It’s a popular hack in the open-weights community.
Why it works: when two models share the same base, their weight differences often *are* their specialized skills — so blending the differences blends the skills.