Vector database

Once you have turned your documents into embeddings, you need to answer one question fast: which of these vectors is closest to the query vector? A vector database is the index built for that question.

What it actually does

Comparing a query against every stored vector is exact but linear. Vector databases trade a little accuracy for a lot of speed using approximate nearest neighbour search, which prunes most of the corpus without comparing against it. They also handle the surrounding plumbing: metadata filters, updates and deletes, and persistence.

The part people skip

For a few thousand chunks you do not need one:

import { cosineSimilarity } from 'ai'

// exhaustive search over an in-memory array
function search(queryVector, docs, k = 5) {
  return docs
    .map((d) => ({ ...d, score: cosineSimilarity(queryVector, d.embedding) }))
    .sort((a, b) => b.score - a.score)
    .slice(0, k)
}

That is exact, has no infrastructure, and is fast enough that you will not notice it. Thousands of vectors compared in memory is not a bottleneck. Reach for a vector database when the corpus outgrows memory, needs concurrent writes, or needs filtered search at scale, not because the tutorial used one.

What to look at when you do need one

• Metadata filtering. Restricting to one tenant, project, or date range is usually the feature you need most, and implementations differ a lot in how well they combine filtering with search.
• Update and delete semantics. Corpora change. Some indexes make deletion expensive.
• Whether it does keyword search too, which decides whether hybrid search is one system or two.