AI 101

CPU inference

Also known as: CPU-only · CPU mode

Local

CPU inference runs the model on the processor with no GPU. It’s slower per token, but it always works.

  • No VRAM requirement.
  • Speed depends on RAM bandwidth and core count.
  • Fine for 3–8B models on a modern CPU.

Don't underestimate it: a 7B Q4 model on a decent CPU is genuinely usable for chat. It's the "it just works" baseline for local AI.