WEBVTT

NOTE AI Engineering Visualised: https://understandingdata.com/ai-engineering-visualised/history-versus-memory/

1
00:00:00.253 --> 00:00:10.651
Every chat reaches the model as a list of messages: a system message from the developer sets the rules, the user asks, and the assistant replies.

2
00:00:10.651 --> 00:00:17.751
The model keeps no memory between calls, so every call sends this whole list, system message first.

3
00:00:17.751 --> 00:00:24.684
Call one sends just the system message and the first user message, and the model promises snake_case.

4
00:00:24.684 --> 00:00:29.970
Then the window empties: the model sees only what the application puts in it.

5
00:00:29.970 --> 00:00:44.306
So the application resends the whole conversation on every call, and seven turns in it comes to 417 tokens, against a window we have shrunk to 300 so you can watch it fill.

6
00:00:44.306 --> 00:00:55.502
Ask for the order type while the first turn still fits, at 192 tokens, and all five runs answer in snake_case, with the total named in pence in four.

7
00:00:55.502 --> 00:01:07.863
Here is the failure mode: after turn seven the application keeps only the newest turns that fit, the first three drop out silently, and all five runs switch to camelCase.

8
00:01:07.863 --> 00:01:23.754
Memory is the obvious fix, yet it is only text the application picks to resend: similarity brought snake_case back in all five runs, but ranked the pence fact last, at 0.06, so it never arrived.

9
00:01:23.754 --> 00:01:34.494
Pinning the conventions that must always hold keeps snake_case in five of five runs, yet only two named the total in pence, so check the output too.

10
00:01:34.494 --> 00:01:45.291
So pin what must persist, retrieve the rest on purpose, and watch the budget, because truncation is silent: the model never knows what it did not see.