Inference
Also known as: running a model · generation
Inference is simply using a trained model: you give it text, it gives you text back. That’s it — every chat, completion, translation and code fix you’ve ever triggered from a chat box was inference.
The easy way to remember the two phases:
| Phase | What it is | Who pays the compute bill |
|---|---|---|
| Training | school — the model learns | the AI company, once |
| Inference | the exam — the model answers | you (or your company), every request |
What happens under the hood: the model reads your tokens and generates the answer one token at a time — each new token chosen from a probability distribution the model computes. That loop, repeated hundreds or thousands of times, is what you watch as a streaming answer.
Why local inference feels slow: your GPU does in one second what a data center does in a millisecond. The model is doing real math for every single token — there's no trick to make it "free", only to make it cheaper (see [quantization](/glossary/quantization/)).