LLM-as-judge
Also known as: LLM as a judge · judge model
LLM-as-judge (LLM-as-a-judge) is the practice of using one model to evaluate another’s outputs — scoring, comparing, or ranking answers automatically.
It’s cheap and scalable, which is why it’s everywhere now. But the judge is itself a model, so:
- It has its own biases (e.g., preferring longer or its own style).
- It can be inconsistent or gamed.
- You must calibrate it against human judgment.
Use it carefully: great for fast, large-scale screening and regression checks — but don't treat the judge as ground truth. Anchor it to a human sample.