Summary
Contextual retrieval prepends a short, generated line of context to each chunk before embedding it, so a chunk that is opaque on its own becomes findable. The chunk you store and the chunk a human would understand are not the same object, and embeddings only know what is in the text.
The Problem
Chunking cuts a document into retrievable pieces, and cutting severs the thread that made each piece legible. A sentence like “It climbed 3% on the prior period” is perfectly clear in place and useless in isolation. The pronoun points at nothing. Embed it raw and the vector carries no signal about revenue, growth, or the company, because none of those words survive in the chunk.
So the query “what was the percentage revenue growth?” reaches for a chunk that says “revenue” out loud, which is usually the wrong one, the sentence stating the level rather than the change. The answer is sitting right there in the index and cannot be found, because the words that make it an answer were three sentences upstream.
The Solution
Before embedding a chunk, ask a cheap model to write one line situating it in its parent document, then embed the context line together with the chunk.
for (const c of chunks) {
const { text } = await generateText({
model: openai('gpt-5-mini'),
prompt: `Document: ${doc}\nChunk: "${c}"\nWrite one short sentence of context for where this chunk sits.`,
})
situated.push(`${text.trim()} ${c}`)
}
const { embeddings } = await embedMany({ model: emb, values: situated })
Now the opaque growth sentence gets embedded as something like “Acme’s revenue rose. It climbed 3% on the prior period,” and its vector finally carries the subject the raw chunk was missing. In the runnable example the target chunk’s similarity to the query rose from 0.543 to 0.607 after situating. On a toy three-chunk corpus the rank may not move, but that lift is precisely what pulls an opaque-but-relevant chunk above the fold once real distractors crowd the index.
The Cost, Stated Honestly
This runs an LLM call per chunk at index time. That is not free, and for a million-chunk corpus it is a real bill. Two things make it pay:
- It is a one-time, offline cost. You situate on ingest, not on every query. Retrieval stays a plain vector lookup.
- Prompt caching collapses most of it. The parent document is identical across all its chunks, so cache the document prefix and you pay full price once per document, not once per chunk.
Contextual retrieval moves work from query time, where latency is precious and repeated, to index time, where it happens once and can be cached. That is almost always the right trade.
When To Reach For It
- Your chunks are short and reference-heavy: pronouns, “the above,” “this figure,” section-relative language.
- Retrieval quality is your bottleneck and your chunks are the reason, not your query or your ranking.
- Skip it when documents are already self-contained (FAQs, product cards, atomic facts) where each chunk stands alone. Adding context there is spend with no lift.
Pair it with hybrid search and a reranking pass and you have covered the three failure points of a RAG stack: what can be found, how the channels are merged, and how the shortlist is ordered.
Related
- Hierarchical Context Patterns – organising context at multiple levels
- Hybrid Search – fuse lexical and semantic retrieval
- Reranking – order the shortlist after retrieval
- Progressive Disclosure of Context – load context only when needed
- Prompt Caching Strategy – what makes per-chunk context affordable
Part of the field guide
This is one of my field notes in Context Engineering, a plain-English guide to deciding what a model sees. The terms behind it are defined in the Context Engineering Dictionary. Runnable version: 13-contextual-retrieval.mjs in my AI Engineering Examples repo.
