Reranking

Vector search is fast because it compares pre-computed embeddings. The document and the query are embedded separately and never seen together, which is exactly why it is quick and also why it is imprecise: nothing ever read the query and the passage side by side.

Reranking adds a second stage that does. Retrieve 50 candidates cheaply, then score each one against the query with a model that reads both, and keep the best 5.

The shape

// stage 1: cheap and generous
const candidates = await vectorSearch(query, { limit: 50 })

// stage 2: expensive and accurate, on 50 items rather than 50,000
const scored = await rerank(query, candidates)   // reads query + passage together
const context = scored.slice(0, 5)

The economics only work because stage 2 runs on a shortlist. A cross-encoder over your entire corpus would be unaffordable; over 50 candidates it is a rounding error.

Why it matters more than it sounds

• Recall and precision are different problems. Stage 1 optimises for not missing anything. Stage 2 optimises for putting the right thing first. Trying to do both in one step gets you neither.
• Position matters inside the window. Because of lost in the middle, the *order* you pass passages in changes the answer. Reranking is how you decide that order on purpose.
• You can retrieve more, not less. Reranking makes a wide stage-1 net safe, which is what makes hybrid search practical.

Choosing the cut

Two numbers to tune: how many candidates you fetch, and how many survive. Fetching more improves the ceiling on quality and costs stage-2 latency. Keeping more fills the window with lower-confidence material. The common failure is keeping too many because the window has room, which pushes your best passage into the middle where the model attends to it least.