Scaling laws

Scaling laws are the empirical observation that model quality improves in a smooth, predictable way as you increase three things: parameters, data, and compute.

This was the quiet engine behind the modern era — it meant “throw more at it” was a reliable strategy, not a gamble.

• Bigger model → better, up to the data limit.
• More data → better, up to the model limit.
• More compute → the budget that buys the above.