Summary
Hybrid search runs lexical retrieval (BM25) and semantic retrieval (vectors) side by side, then fuses the two rankings. Exact-keyword matching and meaning matching fail in opposite directions, so running both and merging beats betting the whole stack on one.
The Problem
Vector search is the default now, and it quietly loses a category of queries: the ones that hinge on an exact token. Error codes, function names, SKUs, OOMKilled. Embeddings smear these into a neighbourhood of “sounds related,” so a search for a precise identifier drifts to paraphrases and misses the literal hit.
Pure keyword search has the mirror flaw. It cannot bridge vocabulary. Search “my process was killed for using too much memory” and BM25 never reaches a document that only says OOMKilled, because they share no words. Each method is blind exactly where the other sees.
The Solution
Retrieve twice, then fuse.
- Lexical. BM25 over a tokenised corpus. It rewards rare shared terms and is unbeatable on exact identifiers.
- Semantic. Embeddings plus cosine similarity. It bridges synonyms and paraphrase.
- Fuse with Reciprocal Rank Fusion. Do not try to compare a BM25 score against a cosine score, they live on different scales. RRF ignores the raw scores and uses rank position only: each document scores
1 / (k + rank)in each list, and you sum. A document that ranks well in either channel floats up; a document that ranks well in both wins.
const rrf = (rank, k = 60) => 1 / (k + rank)
const fused = docs
.map((_, i) => ({
i,
score: rrf(lexical.findIndex((x) => x.i === i)) + rrf(semantic.findIndex((x) => x.i === i)),
}))
.sort((a, b) => b.score - a.score)
One detail the runnable example makes concrete: drop stopwords before BM25. On a small corpus a filler word like “was” is rare, so it earns a high IDF and hijacks the ranking. My first run put an invoice document on top of a memory query purely because both contained “was.” Real lexical engines strip stopwords for exactly this reason, and the toy one has to as well.
Reciprocal Rank Fusion, and Why Not Just Add Scores
The temptation is to normalise both scores to 0-1 and average them. It seems principled and it is fragile: cosine distributions and BM25 distributions shift per query, so any fixed normalisation drifts. RRF sidesteps the whole problem by throwing away magnitudes and keeping order. It is almost embarrassingly simple, has one tuning constant, and outperforms most score-blending schemes people build to replace it.
When To Reach For It
- Your corpus mixes prose with identifiers: code, logs, product catalogues, legal or medical text with precise terms of art.
- Users sometimes paste exact strings and sometimes describe things in their own words. You cannot predict which.
- Skip the ceremony only when your queries are uniformly natural-language and your corpus has no load-bearing exact tokens. That is rarer than most teams assume.
Hybrid retrieval usually sits underneath a reranking pass: fuse to build a strong shortlist, then rerank to order it. The two patterns are complements, not alternatives.
Related
- Reranking – order the shortlist that hybrid search assembles
- Contextual Retrieval – make individual chunks findable in the first place
- GraphRAG for Production Agents – structured retrieval when flat search is the wrong shape
- Vendor the Source, Skip the Search – when the right answer is to not search at all
- Information Theory for Coding Agents – signal and entropy in retrieval
Part of the field guide
This is one of my field notes in Context Engineering, a plain-English guide to deciding what a model sees. The terms behind it are defined in the Context Engineering Dictionary. Runnable version: 12-hybrid-search.mjs in my AI Engineering Examples repo.
