WEBVTT

NOTE AI Engineering Visualised: https://understandingdata.com/ai-engineering-visualised/prompting-rag-fine-tuning/

1
00:00:00.251 --> 00:00:13.594
Prompting, RAG and fine-tuning each fix a model's answer by changing a different thing: prompting changes the instructions you send, RAG adds evidence to each call, and fine-tuning changes the model itself.

2
00:00:13.594 --> 00:00:32.467
Prompting here is one added line, 39 tokens and instant; RAG fetches our updated refund policy at question time, so the model reads a fact it was never trained on; fine-tuning trains on past replies, which changes style and behaviour, but is slow to redo.

3
00:00:32.467 --> 00:00:47.503
To see the difference, take one request: Priya asks for a refund forty days after paying, and the bare model fails three ways at once: prose instead of JSON, an invented 30-day rule, and none of our house style.

4
00:00:47.503 --> 00:00:59.062
Lane one edits the instructions: one added line, 39 more tokens, and all five runs return valid JSON, yet every one still declines on 30 days.

5
00:00:59.062 --> 00:01:11.546
Lane two leaves the instructions alone and inserts the updated policy instead, and all five runs quote 45 days and grant the refund, but back in prose, and still not in our voice.

6
00:01:11.546 --> 00:01:25.967
Lane three changes the weights: a rank 8 adapter trains 0.12 percent of a 7B model, and here forty past replies stand in for that run, so all five replies now sound like our team.

7
00:01:25.967 --> 00:01:36.371
Now read the scoreboard as a diagonal: each fix repairs only its own fault, so the question is never which technique wins, but which kind of fault you have.

8
00:01:36.371 --> 00:01:49.059
And here is the trap: those past replies still carried the old 30-day rule, so lane three declines Priya in perfect house style, because facts trained into weights go stale.

9
00:01:49.059 --> 00:02:04.710
So match the fix to the fault, instructions for shape, retrieval for facts and training for behaviour, and together all five runs pass, though the examples cost 2,685 tokens a call until you train them in.