🦙 Ollama: run LLMs locally in 3 commands

What Ollama actually is

Ollama is a small program that does three things:

1. Downloads models for you (pre-quantized, ready to run).
2. Runs them locally, on your CPU or GPU, and keeps exactly one loaded in memory.
3. Exposes a local API on http://127.0.0.1:11434 — so any app that can speak HTTP (or the OpenAI format) can use your model.

That third point is the big deal. The chat you’ll do in the terminal in a minute is the toy; the API is what makes local AI useful.

Step 1 — Install

On Windows, the easiest path is the installer (or winget):

winget install Ollama.Ollama

Check it works:

$ ollama --version
ollama version is 0.40.15

That’s it — Ollama installs itself as a background service on Windows, so the server is already running. No extra “start” command needed.

Step 2 — Pull a model

Models are stored as GGUF files, pre-quantized. A good first pick: 4B parameters, Q4_K_M — small enough for most laptops, smart enough to be useful.

$ ollama pull qwen3:4b
pulling e18a783aae55: 100% ▕██████████████▏  487 B
verifying sha256 digest
writing manifest
success

That’s a ~2.6 GB download — it took about 13 minutes on a normal broadband line. Once it’s on disk, it’s free and offline forever.

See what you have:

$ ollama list
NAME           ID              SIZE      MODIFIED
qwen3.8:27b    aaee06c39dcf    17 GB     12 days ago
qwen3.5:9b     56671c2ab938    6.6 GB    2 weeks ago
qwen3:4b       359d7dd4bcda    2.6 GB    just now

The SIZE column is the download size. The loaded size in RAM/VRAM is bigger (weights + context), which the next step shows.

Step 3 — Chat with it

$ ollama run qwen3:4b "Réponds EXACTEMENT avec ce texte et rien d'autre : FOUR_OK"
FOUR_OK

Yes, that’s the whole tutorial. Everything below is what makes it usable.

The local API (the useful part)

While a model is loaded, Ollama serves HTTP on port 11434. Two dialects:

Native format:

$ curl http://127.0.0.1:11434/api/chat -d '{
    "model": "qwen3:4b",
    "messages": [{"role": "user", "content": "Réponds EXACTEMENT : API_OK"}],
    "stream": false
  }'

OpenAI-compatible format — this is why every app in existence can point at Ollama:

$ curl http://127.0.0.1:11434/v1/chat/completions \
  -d '{"model":"qwen3:4b","messages":[{"role":"user","content":"Hi"}]}'

Point any OpenAI SDK at base_url = "http://127.0.0.1:11434/v1" and your app now runs on the model in your machine instead of in a datacenter. Same code, different base_url.

The daily commands

Two things in that ps output are worth understanding:

• 100% GPU — if you have a decent NVIDIA/AMD/Apple GPU, Ollama put the model there. On CPU-only machines this will say 100% CPU and generation will be much slower.
• UNTIL 4 minutes from now — Ollama auto-unloads models after ~5 idle minutes to free memory. It’s not a bug: the next run just reloads it (a few seconds).

What to expect from your hardware

Rule of thumb: model file size × ~1.5–2 ≈ what you need in RAM/VRAM, context included. The Q4 quantization is what makes this possible — a 27B model that would need 55 GB in full precision fits in 17 GB. See why →

Where this leaves you

You now have a private, offline chat model and a local API. The natural next step is a proper interface on top of it: