Data quality
Also known as: data curation
Data quality is how clean, accurate, consistent, and relevant the training examples are. It’s frequently the single biggest lever on how a model behaves.
Good curation removes:
- Duplicate and boilerplate text.
- Low-quality or toxic content.
- Content that would teach the model bad habits.
A small amount of good data can outperform a huge amount of noisy data.
The underappreciated truth: much of the difference between models isn't the architecture — it's how carefully the data was chosen and cleaned.