llama.cpp
Also known as: llama cpp · llamacpp
llama.cpp is a C/C++ library that runs LLM inference on ordinary hardware — no special GPU required. It’s the engine under most local tools.
- Reads GGUF files.
- Runs on CPU, or with GPU offload (CUDA/Metal).
- Fast, small, dependency-light.
The local backbone: Ollama, llamafile, and dozens of apps are wrappers around llama.cpp. Understanding it explains most of the local ecosystem.