WEBVTT

NOTE AI Engineering Visualised: https://understandingdata.com/ai-engineering-visualised/vector-search/

1
00:00:00.047 --> 00:00:15.531
Vector search means finding the stored vectors nearest a query, and the obvious way compares the query with every single one: here that is 500 distance computations, and a billion vectors cost a billion.

2
00:00:15.531 --> 00:00:29.654
HNSW builds a tower instead: every point lives on the ground floor, a lucky few also climb higher, so the roof holds just nine points joined by long links, while the ground floor links close neighbours.

3
00:00:29.654 --> 00:00:46.109
Our query enters at the roof and hops to closer neighbours, dropping a floor when none is closer: six hops reach the ground, where a short search finds the same ten neighbours as a full scan, with 63 comparisons instead of 500.

4
00:00:46.109 --> 00:01:06.052
The catch is that the walk can get lost: on this flat floor it rarely does, but in 32 dimensions, closer to real embeddings, an ef of ten finds about half of the true ten, and 94 percent takes an ef of 64, at 217 comparisons.

5
00:01:06.052 --> 00:01:24.378
Text adds a second trap: embeddings score meaning as the angle between rays, so for error E1417 they rank E1147's page first, 0.64 to 0.61, while BM25 matches the exact code.

6
00:01:24.378 --> 00:01:39.766
A paraphrase breaks the other way: asked how to get my money back, BM25 matches money to Payouts and back to Cancelling, and never finds Refunds, which shares no word, yet embeddings rank it first.

7
00:01:39.766 --> 00:02:04.082
Reciprocal rank fusion merges two lists by rank alone: a page earns one over sixty plus its rank from each list, so Payouts, third by angle and first by keywords, scores 0.0323, while Refunds, first by angle but missing from BM25, gets 0.0164.

8
00:02:04.082 --> 00:02:23.625
That sixty is k, and it is the dial: at 60, second place is worth 98 percent of first, so turning up in both lists beats winning one; set k to zero and first place counts double, and Refunds climbs from fourth to second.

9
00:02:23.625 --> 00:02:41.981
So run both, but fuse with care: no k splits E1417 from E1147, because each list puts one first and the other second, so a reranker belongs after the fusion, and you measure recall on your own questions.