AI engineering glossary · RAG vector search
What is reranking?
Reranking: A second stage that takes the candidates from the first retrieval and reorders them using a more accurate but slower model. It improves the precision of the few passages finally sent to the language model.
Reranking: an example
Retrieve 50 candidates with vectors, rerank them with a cross-encoder, and pass the best 5 to the generator.
Reranking in Python
A short example you can run (Python 3.11). The output below is what it really prints.
candidates = [
"GST registration is needed above the turnover threshold",
"Our refund window is 7 days",
"Refunds are processed to the original payment method within 7 days",
]
query = "how long do refunds take"
def overlap_score(q, d): # stand-in for a cross-encoder model's relevance score
qs, ds = set(q.lower().split()), set(d.lower().split())
return len(qs & ds) / len(qs)
for d in sorted(candidates, key=lambda d: -overlap_score(query, d)):
print(round(overlap_score(query, d), 2), d)Output
0.2 Refunds are processed to the original payment method within 7 days 0.0 GST registration is needed above the turnover threshold 0.0 Our refund window is 7 days
Related terms
More RAG vector search terms
- Rescoring
- Retrieval evaluation
- Retrieval-augmented generation
- Retriever
- Scalar quantisation
- Score fusion
- Self-RAG
- Semantic cache
Where you learn it
Reranking is part of the AI Engineering Program, which teaches it with notes, a hands-on lab and an exam. The first lesson is free.