VRAM
Also known as: video memory · GPU memory
VRAM (video RAM) is the dedicated memory on a GPU. For local LLMs, the rule is simple:
- Model weights + KV cache must fit in VRAM to be fast.
- What doesn’t fit spills to system RAM (slow) or is offloaded.
A 7B model at Q4_K_M needs roughly 4–5 GB; a 13B needs ~8–10 GB; a 70B won’t fit in one consumer card.
The #1 local constraint: VRAM, not compute, is usually what stops you running a bigger model. Check it first.