Hybrid search optimisation guide: BM25 + vectors + reranking

Short answer: hybrid search runs a keyword search (usually BM25) and a vector (embedding) search for the same query and merges the two ranked lists, most simply with reciprocal rank fusion (RRF). Keywords catch exact terms such as IDs, codes and names; vectors catch meaning and paraphrases. Tune it by measuring recall@k on real queries, then add a reranker over the merged top results.

Why one kind of search is not enough

• Vector search alone misses exact matches: an invoice number, "Form 16", a product SKU or a rare surname may not sit close to the query in embedding space.
• Keyword search alone misses meaning: "money back" will not match a passage that only says "refund".

Real questions mix both, which is why hybrid search is the default in most production RAG systems.

Step 1: two retrievers

Index the same chunks twice: a BM25 index (Elasticsearch/OpenSearch, Postgres full-text search, or a library such as rank_bm25) and a vector index (pgvector, Qdrant, Chroma...). Retrieve the top 20 to 50 from each. A from-scratch BM25 scorer you can run is on the BM25 glossary page.

Step 2: merge with reciprocal rank fusion

RRF ignores the raw scores (which are on different scales) and uses ranks only: score(d) = sum over lists of 1 / (k + rank(d)), with k around 60. A document that ranks well in both lists rises to the top. Runnable code with its output is on the hybrid search glossary page. If one retriever is clearly better for your data, weight its contribution higher.

Step 3: rerank

Send the merged top 20 to 50 to a cross-encoder reranker, which reads the query and each passage together and scores relevance far better than either retriever, then keep the best 3 to 8 for the model. Reranking adds latency, so measure it.

Step 4: tune with data

1. Collect 50 to 200 real queries with their relevant passages (from logs, support tickets or people who know the documents).
2. Measure recall@k and MRR for keyword only, vector only and hybrid.
3. Tune one thing at a time: k in RRF, the weights, how many results each retriever returns, the chunk size.
4. Watch per-query failures, not only averages. Exact-match queries and paraphrase queries often move in opposite directions.

Practical tips

• Normalise text the same way at index and query time (case, Unicode, numbers like "₹1,000" vs "1000").
• For Indian-language or mixed English-Hindi queries, check that both the tokenizer and the embedding model handle the script; multilingual embeddings help, and query translation is a useful fallback.
• Add metadata filters (document type, date, language) before ranking when the user's intent makes them clear.

Where this is taught

Module 8 (RAG Engineering): embeddings and vector databases, hybrid search, reranking, query rewriting and routing, multilingual RAG for Indian languages and measuring retrieval. Related: Fixing RAG context fragmentation · Reranking.