Embeddings & vector search
How meaning becomes geometry — the quiet technology under semantic search, RAG, and 'find me something similar'.
The problem: computers can’t compare meaning
Ask a normal program: “which of these paragraphs is most similar to this one?” — and it can only do word matching. “Car” ≠ “vehicle”, “sad” ≠ “miserable”, and any paraphrase breaks it. Text search is a spelling game.
Embeddings fix this by turning meaning into geometry.
The idea in one picture
An embedding model reads a piece of text and outputs a vector — a list of numbers, typically 768 to 3,072 long:
"the cat sat on the mat" → [0.12, -0.44, 0.87, …]
"the kitten lounged on the rug" → [0.14, -0.41, 0.82, …] ← close!
"quarterly revenue increased" → [-0.63, 0.22, -0.91, …] ← far
Two sentences with similar meaning land near each other in this high-dimensional space; unrelated ones land far apart. “Similar” is now just distance — and distance is something computers are very good at computing.
The operation: cosine similarity
The standard measure isn’t “how far apart” but “how aligned” the two vectors point — called cosine similarity. Same direction ≈ same meaning, regardless of “size” (length, style). It’s the one metric you’ll meet everywhere in this space, and it’s the only one you need to remember.
Why this is a big deal
Embeddings are the glue between text and everything else:
| Use | What it unlocks |
|---|---|
| Semantic search | “Find documents about pricing” — even if the word pricing never appears |
| RAG | Retrieve the right chunks → feed them to the LLM → grounded answers |
| Deduplication | “These 1,000 support tickets, how many are actually different problems?” |
| Recommendation | “More of this” becomes a nearest-neighbors query |
| Clustering | Group 10,000 reviews into themes without writing a single rule |
The one gotcha
An embedding is a snapshot of meaning at a moment, from one model. Two different embedding models produce different spaces — you can’t mix vectors from different models. And a model that embeds English well doesn’t automatically embed French well (it can, but it’s a separate capability to check).