Tokens per second (tok/s)
Also known as: tok/s · t/s · speed
Tokens per second (tok/s) is how fast a model generates output — the headline number for local inference speed.
- 20–40 tok/s: comfortable reading speed.
- 5–15 tok/s: usable but slow.
- < 5 tok/s: painful for chat.
It depends on model size, quant, and whether it’s on GPU or CPU.
The local KPI: for a usable chat experience, aim for at least ~15 tok/s. That single number tells you more than any benchmark table.