WEBVTT

NOTE AI Engineering Visualised: https://understandingdata.com/ai-engineering-visualised/transformer-block/

1
00:00:00.220 --> 00:00:12.357
A transformer block has one job, to sharpen each word's vector, so first our eight words, from The to purred, look up their rows in the embedding table.

2
00:00:12.357 --> 00:00:26.353
Next, every vector is rescaled to the same length, so mat at 1.06 and The at 0.70 both leave at exactly one before anything reads them.

3
00:00:26.353 --> 00:00:36.781
Then mat looks back across the earlier words, and in this head it spends 0.70 of its attention on the, the word right before it.

4
00:00:36.781 --> 00:00:47.960
Here is the part people miss: attention adds to the vector instead of replacing it, so the original signal survives, layer after layer.

5
00:00:47.960 --> 00:01:02.015
Next, each vector widens fourfold, from 768 numbers to 3,072, where only a few units fire, and then it narrows back down to 768.

6
00:01:02.015 --> 00:01:16.060
Last, after the final block, the vector is scored against all 50,257 words, and when the top two scores sit close together, a tiny change in the prompt flips the answer.

7
00:01:16.060 --> 00:01:25.521
Pulled apart, two thirds of every block's weights sit in the feed-forward layer, so that is where quantising a model saves the most.