vLLM
Also known as: vllm
vLLM is a fast LLM serving engine that uses PagedAttention to manage the KV cache like virtual memory — far less wasted GPU memory.
- High throughput, good batching.
- The default choice for serving models in production.
- Works with HF models out of the box.
When you need it: when "run one model on my laptop" becomes "serve many requests to many users", vLLM is the standard answer.