What is BM25? Meaning and example in Python

What is BM25?

BM25 (also called Okapi BM25): A ranking function for keyword search that scores how well a document matches a query. It builds on term frequency and inverse document frequency, but limits the gain from repeated terms and adjusts for document length. It remains a strong baseline for search.

Level: intermediateTopic: NLP embeddings

BM25: an example

Searching "solar subsidy India" ranks a short government page that repeats these words above a long article mentioning them once.

BM25 in Python

A short example you can run (Python 3.11). The output below is what it really prints.

import math

docs = [d.split() for d in ["gst invoice format", "gst rate on courses", "pytorch gpu setup"]]
avgdl = sum(map(len, docs)) / len(docs)

def bm25(query, doc, k1=1.5, b=0.75):
    score = 0.0
    for q in query.split():
        n = sum(q in d for d in docs)
        idf = math.log((len(docs) - n + 0.5) / (n + 0.5) + 1)
        tf = doc.count(q)
        score += idf * tf * (k1 + 1) / (tf + k1 * (1 - b + b * len(doc) / avgdl))
    return round(score, 3)

for d in docs:
    print(" ".join(d), bm25("gst invoice", d))

Output

gst invoice format 1.519
gst rate on courses 0.431
pytorch gpu setup 0.0

Related terms

More NLP embeddings terms

Where you learn it

BM25 is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.