Embeddings and similarity search

Concept#

An embeddingEmbeddingA list of numbers (a vector) that represents a token, chunk or query so that similar meanings sit close together. Used for search, clustering and as the model's input.Open the glossary entryClose is a list of numbers (a vector) that a model produces for a piece of content, so that content with similar meaning gets vectors that point in similar directions. Text, code and images can all be embedded.

This is a different thing from the per-token embedding table inside an LLM, which Tokenization and embeddings covers. Here a dedicated embedding modelEmbedding modelA model whose job is to turn text into one fixed-length vector. Index and query must use the same one, because each model builds its own vector space.Open the glossary entryClose reads a whole chunk, document or query and returns one vector for all of it. This lesson is about what you can do with those vectors: compare them, rank by them, and know when that ranking fails.

Why It Matters#

Embedding search is the retrieval half of every RAG system, most semantic search products and many recommenders. It is also easy to misuse. Teams ship it as “fuzzy keyword search”, then are surprised when it fails on a product code or returns pages on the right topic that do not answer the question. Knowing the geometry tells you which failures to expect and which fix to reach for.

Mental Model#

Think of embeddings as map coordinates for meaning, with hundreds or thousands of axes instead of two. “Dog” and “puppy” land close together. “Dog” and “spreadsheet” land far apart, whatever letters they share.

Searching is then “drop a pin at the query’s coordinates and list the nearest pins”. Everything that follows is about how to measure “nearest” and what the map cannot show.

Example#

A user types “how do I cancel my subscription”. The right help page is titled “Ending your plan”. The two share almost no words, so keyword search misses it. An embedding model places both near the same region of the space, because it learned that these phrases are used in the same situations. The page ranks high.

Technical Deep Dive#

How the model learns “close”#

Embedding models are typically trained with contrastive learningContrastive trainingTraining that pulls matching pairs of texts together and pushes others apart in vector space. It is why close vectors tend to mean similar meaning.Open the glossary entryClose. The model sees pairs that should match (a question and its answer passage, two paraphrases) and pairs that should not. Training pulls the matching pairs together and pushes the others apart. “Meaning” here means whatever the training pairs made similar, which matters later when you hit domain shift.

Dimensions cost storage#

A model outputs a fixed number of dimensions. More dimensions can hold more nuance, but every vector costs storage and compute. As an example, 1 million vectors of 1,536 dimensions stored as 32-bit floats take 1,000,000 x 1,536 x 4 bytes = about 6.1 GB, before any index overhead. Some providers let you request shorter vectors, with a small quality cost. OpenAI’s embedding guide, for instance, lists 1,536 default dimensions for text-embedding-3-small and 3,072 for text-embedding-3-large, and a dimensions parameter to shorten them (checked 2026-10-09). Model names and sizes change often, so check the model you use.

Three ways to measure “close”#

Cosine similarity is cos = a·b / (|a| |b|), the dot product divided by the two lengths.

Why normalizing makes cosine cheap#

If both vectors are scaled to length 1 (unit normVector normalizationScaling a vector to length 1. After it, cosine equals the dot product, which is cheaper to compute.Open the glossary entryClose), the denominator is 1, so cos = a·b. Cosine becomes a plain dot product, which hardware does very fast. With unit vectors, Euclidean distance gives the same ranking too, because |a - b|² = 2 - 2(a·b).

A worked example with two unit vectors in 2 dimensions: a = (0.6, 0.8), b = (0.8, 0.6). Their dot product is 0.48 + 0.48 = 0.96, so cosine is 0.96. Their squared distance is 0.04 + 0.04 = 0.08, and 2 - 2 x 0.96 is also 0.08. Same ordering, two different formulas.

Some providers return already-normalized vectors. OpenAI’s guide says its embeddings are length 1, so cosine can be computed as a dot product (checked 2026-10-09). Other models may not be. Check the model card, and use the metric the model was trained for. The vector index must be configured with that metric (Vector databases).

Diagram#

Code#

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

a = np.array([0.6, 0.8])
b = np.array([0.8, 0.6])
print(cosine_similarity(a, b))  # 0.96; equals np.dot(a, b) because both have length 1

Where embedding search fails#

These are the failure modes to check first when results look wrong.

• Exact identifiers and numbers. A query such as ERR-40412 is a string of characters with little “meaning”. The model may return pages about errors in general. Fix: add keyword search (BM25) next to vector search and merge the results. This is called hybrid searchHybrid searchRunning keyword search (BM25) and vector search together and merging the lists. It covers exact terms and paraphrases.Open the glossary entryClose, covered in Better retrieval.
• Rare terms and new vocabulary. Product names, internal jargon and new acronyms may be missing from the model’s training data, so they land in arbitrary places. Fixes: hybrid search, or a model trained on your domain.
• Domain shift. A general model may treat all legal clauses, or all medical notes, as roughly similar. Fix: measure on your own queries, try another model, and consider fine-tuning only after measuring.
• Related but not useful. Embeddings find text on the same topic, which is not the same as text that answers the question. Fix: retrieve more candidates, then rerank with a model that reads the query and each candidate together.
• Mixed models. Vectors from different embedding models live in different spaces. If you embed documents with one model and queries with another, or you upgrade the model without re-embedding the corpus, scores are meaningless. Fix: store the model name and version with the index, and re-embed everything when you change it.

Real-World Application#

Semantic search, near-duplicate detection, clustering, recommendations and the retrieval step of RAG (RAG architecture and chunking) all use this mechanism. For near-duplicate detection across 1 million documents, you embed each one, then search for neighbors above a similarity threshold. Pick the threshold by labelling a sample of real pairs, not by guessing, because the right score depends on the model.

Interview Perspective#

Expect questions such as “why cosine over Euclidean for text”, “how would you find near-duplicate documents in a corpus of 1M” and “your search returns the wrong document for an error code”. A common mistake is calling embedding search “fuzzy keyword matching”: it is semantic, not lexical, and that is exactly why it fails on exact strings.

Provenance: Common industry pattern. These are standard topics for GenAI roles, not a verified list for any named company.

Practice Questions#

These are generated practice questions, not reported from any company.

• Basic: What does it mean for two embeddings to be “close”?
• Intermediate: Why does normalizing vectors make cosine similarity cheap to compute?
• Advanced: Why would two paraphrased sentences with zero shared words still have high similarity?
• Product-Company: Your semantic search returns results that are on topic but not useful answers. What is the likely gap in the pipeline?

Activity#

Write three pairs of sentences that share no words but mean similar things, and one pair that shares many words but means different things (for example, a negation). Predict which pairs score high before you compute anything. Then compute the scores and explain any surprise.

Homework#

Using the function above and any embedding model you can call, embed three short sentences you write. Compute all pairwise cosine similarities, explain the ranking, then check whether your model’s vectors have length 1 (print np.linalg.norm). Write two sentences on what that means for your choice of metric.

Quick Revision#

• An embedding model maps a whole chunk or query to one vector; close vectors mean similar meaning, as learned from training pairs.
• cos = a·b / (|a||b|); on unit-length vectors it is just a·b.
• Cosine is the usual text default. Check what your model expects, and configure the index to match.
• Exact IDs, rare terms and domain shift are weak spots; hybrid search and reranking address them.
• Index and query must use the same embedding model and version.

Check yourself#

• Walk-through: Explain why cosine similarity on normalized embeddings equals the dot product, and name two failure modes of embedding search that hybrid search addresses.
• Variation: Your query ERR-40412 returns semantically similar but wrong documents. Diagnose it and propose a fix.
• Common misconception: Believing embeddings capture exact-match meaning, or that vectors from two different models can be compared.

Related#

• Tokenization and embeddings
• Vector databases
• Better retrieval
• RAG architecture and chunking