Vector search: HNSW, cosine similarity and BM25

Vector search finds the stored vectors nearest a query vector: the step inside retrieval-augmented generation (RAG), semantic search and recommendation that turns "which documents are about this?" into geometry. The exact answer needs a brute-force scan, one distance computation per stored vector, which is fine for thousands and ruinous for a billion. Approximate nearest neighbour (ANN) indexes such as HNSW trade a little recall for far fewer comparisons, and this film measures that trade rather than asserting it.

The tower is a real HNSW graph (hierarchical navigable small world, Malkov and Yashunin), built by the code on this page over 500 seeded points in a plane with four links per point on the upper floors and eight on the ground. Every point sits on the ground floor; each one climbs to higher floors with shrinking odds, so the floors hold 128, 35 and 9 points and their links stretch across the whole plan. A search enters at the roof, moves greedily to any neighbour closer to the query, and drops a floor when none is. The traced query makes two hops on each upper floor, then a short beam search on the ground floor, and finds exactly the ten neighbours a full scan finds, with 63 distance computations instead of 500. The ef control sets the length of that ground-floor candidate list; pause the film and change it to see the visited points and the counter grow.

The part people get wrong is how much ef matters, because a demo in two dimensions flatters every ANN index. On the flat floor ef 10 already finds 99.5% of the true ten on average. Run the same code on 500 unit vectors in 32 dimensions, closer to how real embeddings behave, and ef 10 finds only 52% of them, ef 64 finds 94% for 217 comparisons, and ef 128 finds 99% for 328. Recall also falls as the collection grows at a fixed ef, so the right setting is not a constant: measure recall at k against a brute-force scan on a sample of your own queries, and re-measure when the index grows.

The second half is text, with real numbers. Twelve help-centre pages and nine questions were embedded with openai/text-embedding-3-small on 11 September 2026, and every page was also indexed for BM25 with Lucene's stop words and defaults. Cosine similarity is drawn as it is defined, the angle between two rays: the protractor puts each page at its measured angle from the question. Asked "payment failed with E1417", the embedding ranks the E1147 page first, 0.64 against 0.61, because the two codes are nearly the same string to a model that reads meaning; BM25 looks the token e1417 up in its inverted index, finds one page, and ranks it first. Asked "How do I get my money back?", the failure reverses: BM25 matches money to Payouts and back to Cancelling an order, and never reaches Refunds, which shares no word with the question, while the embedding ranks Refunds first at 0.42.

Hybrid search runs both and fuses the lists, most often with reciprocal rank fusion, which scores each page by 1/(60 + rank) summed over the lists. On this recording fusion did not pick the winner. It rewards agreement, so it tied E1147 with E1417 and dropped Refunds to fourth behind pages both lists half-liked. Across all nine questions the right page came first 8 times by embeddings alone, 5 by BM25 and 4 by fusion, with 3 more tied at the top, while the fused top four held the right page every time. That is the honest shape of hybrid retrieval: it widens the candidate set cheaply, and a reranker or a model reading the top few does the choosing.

The caveats are about scale. Twelve pages and nine questions show the two failure modes but cannot rank the methods; published benchmarks such as BEIR find hybrid retrieval usually helps on large corpora, and the first corpus tried here, with shorter pages, did not show the exact-code failure at all. The tower is two-dimensional so it can be drawn, and production indexes use larger M and ef, millions of vectors, and filters, deletions and quantisation that this build leaves out. The numbers are nonetheless all computed or recorded, and the tests recompute every one the narration quotes.