Eval (evaluation)
Also known as: evals · evaluation
An eval (evaluation) is a structured test: run a model on a fixed set of prompts/tasks and score the outputs — automatically or with humans.
It’s how teams answer “did our change make the model better or worse?” with evidence.
- Automated evals: fast, repeatable, great for regression.
- Human evals: richer, slower, expensive.
- LLM-as-judge: a model grades another model’s answers (cheap, needs care).
The discipline: good teams eval *before* and *after* every change. "It feels better" is not a measurement — an eval is.