WEBVTT

NOTE AI Engineering Visualised: https://understandingdata.com/ai-engineering-visualised/rag-question-to-answer/

1
00:00:00.260 --> 00:00:12.966
Ask a model how long we keep production database backups and it can only list industry ranges from seven to thirty-five days, because the answer lives in our handbook, not in the model.

2
00:00:12.966 --> 00:00:27.687
So retrieval first embeds the question and scores every handbook passage by similarity: the retention page scores 0.63, the backup overview 0.59, and the monitoring page 0.58.

3
00:00:27.687 --> 00:00:36.612
The top three go into the prompt above the question, under one rule: cite a passage for every claim, or say the answer is missing.

4
00:00:36.612 --> 00:00:47.273
The recorded answer is one sentence, thirty-five days, tagged P2, and that tag is the point: every claim can be checked against the passage it came from.

5
00:00:47.273 --> 00:00:59.248
Now delete the retention page: the next three fill the prompt, the staging page among them, and the model says the passages do not contain the answer, which is the right reply to a retrieval failure.

6
00:00:59.248 --> 00:01:15.295
The part people miss is that similarity rewards wording: an outdated 2023 page whose title copies the question outranks the real one, 0.67 to 0.63, and the answer now repeats its stale fourteen days.

7
00:01:15.295 --> 00:01:19.583
So score retrieval on its own: did the right passage reach the prompt?

8
00:01:19.583 --> 00:01:24.606
A refusal can be a retrieval bug, and a cited answer can still carry a stale page.