Concept guide

Embeddings explained: what they are and how similarity search works

What text embeddings are, how cosine similarity finds related meaning, a tiny worked example in Python, and where embeddings are used (search, RAG, recommendations).

2 min readFree, no sign-up

Short answer: an embedding is a list of numbers (a vector) that represents the meaning of a piece of text, an image or other data. Items with similar meaning get vectors that point in similar directions, so a computer can find "related" items by measuring the distance or angle between vectors instead of matching exact words.

Why not just match words?

A keyword search for "cheap flights" misses a page titled "low-cost air tickets". An embedding model maps both phrases to nearby vectors, so a similarity search finds it. This is the engine behind semantic search, recommendations and the retrieval step of RAG.

How similarity is measured

The common measure is cosine similarity: the cosine of the angle between two vectors. It is 1 when they point the same way, near 0 when unrelated. When vectors are normalised to length 1, it equals the dot product. Glossary pages: semantic similarity and dot-product similarity.

A tiny worked example

Real embeddings have hundreds or thousands of numbers; here are three, to show the idea.

import numpy as np

cat = np.array([0.9, 0.1, 0.0])
kitten = np.array([0.8, 0.2, 0.1])
car = np.array([0.1, 0.9, 0.3])

def cos(a, b):
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

print(round(cos(cat, kitten), 2), round(cos(cat, car), 2))

This prints 0.98 0.21: "cat" is close to "kitten" and far from "car".

Where you meet embeddings

  • Semantic search: embed the query and every document, return the nearest.
  • RAG: retrieve the passages nearest to a question before the model answers.
  • Recommendations and deduplication: find similar products or near-duplicate tickets.
  • Clustering and classification: group items by meaning.

Practical points

  • Use the same embedding model for documents and queries.
  • Store vectors in a vector database or an index with approximate nearest-neighbour search, because exact search is slow at scale.
  • Embeddings capture topic well but can miss exact identifiers (part numbers, names), so many systems combine them with keyword search: see hybrid search.
  • Chunk size matters: very long passages blur meaning. See chunking problems.

Words used on this page

Text embedding model · Sentence embedding · Semantic similarity · Vector database