Hybrid search

Embeddings are good at meaning and bad at literals. Ask a vector index for ECONNREFUSED and it will happily return three chunks about network reliability and miss the one line containing the actual string. Keyword search has the opposite problem: it finds the exact token and misses the paragraph that describes the same idea in different words.

Hybrid search stops you choosing. Run both, then merge.

Merging with reciprocal rank fusion

The tricky part is combining two lists whose scores are not comparable: a BM25 score of 12.4 means nothing next to a cosine similarity of 0.83. Reciprocal rank fusion sidesteps this by throwing the scores away and using only the rank each list assigned:

function fuse(lists, k = 60) {
  const scores = new Map()
  for (const list of lists) {
    list.forEach((id, i) => {
      scores.set(id, (scores.get(id) ?? 0) + 1 / (k + i + 1))
    })
  }
  return [...scores.entries()]
    .sort((a, b) => b[1] - a[1])
    .map(([id]) => id)
}

// keyword ranked C first, vectors ranked D first, both put B second
fuse([['c', 'b', 'a'], ['d', 'b', 'a']])
// => ['b', 'a', 'c', 'd']

Note what happened to b. It was never first in either list, yet it comes out on top, ahead of both c and d which each won one list outright. Appearing in both lists scored it 0.032, while a single first place scored only 0.016. That is the whole point: agreement across two different retrieval methods beats winning either one alone.

When it earns its keep

• Code and logs. Identifiers, error strings, and flags are exactly what vector search fumbles.
• Product and proper nouns. A model name or SKU needs to match literally.
• Mixed corpora, where some documents are prose and some are structured.

If your corpus is pure prose and users ask conceptual questions, plain vector search is usually enough and hybrid is extra machinery for little gain.