Toxicity
Also known as: toxic output · harmful output
Toxicity is a model producing abusive, hateful, or otherwise harmful content.
- A failure category in safety evals.
- Target of safety filters and red teaming.
- Context matters: a word is fine in one setting, toxic in another.
Measure it: include a toxicity check (or a small held-out set of "this should be refused" prompts) in any eval you run. It's the cheapest proxy for a class of real harm.