Data mixing
Also known as: data mix · mixture
Data mixing is the choice of what fractions of each data source go into the training mix — how much web text, code, math, books, multilingual content, and so on.
It directly shapes what the model gets good at:
- More code in the mix → better coding.
- More multilingual text → better non-English.
- More math/reasoning → stronger problem-solving.
It’s a deliberate engineering decision, not an accident.
Why models differ: two models of the same size can behave differently mainly because of *what they were fed and in what proportions*. Data mixing is a core differentiator.