WEBVTT

NOTE AI Engineering Visualised: https://understandingdata.com/ai-engineering-visualised/softmax/

1
00:00:00.267 --> 00:00:17.440
Every step ends with one raw score per candidate token, called a logit: here Paris scores four, Lyon two, France one and a half, and the word the minus one, and none of that is a probability yet.

2
00:00:17.440 --> 00:00:35.325
So first each score goes through the exponential, which makes every value positive and turns Paris's two point lead over Lyon into 54.6 against 7.4, about seven times, and real code subtracts the largest score first so nothing overflows.

3
00:00:35.325 --> 00:00:45.018
Then divide each one by their sum, 66.8, and the shares add to exactly one: Paris owns 82 percent.

4
00:00:45.018 --> 00:00:56.557
The part people get backwards is the dial: temperature divides the logits before the exponential, so below one it sharpens the shares and above one it flattens them.

5
00:00:56.557 --> 00:01:08.115
Training scores only the share on the right answer, so being confident and wrong is what costs: make Lyon the answer, at 11 percent, and the gauge jumps past three bits.

6
00:01:08.115 --> 00:01:25.637
When one model is distilled into another, KL divergence compares their whole distributions: each bar is a token's share times its log ratio, and swapping which tube does the believing changes the total, because KL is not a distance.

7
00:01:25.637 --> 00:01:37.003
The takeaway is perplexity: this model is as unsure as a choice between 1.9 equal options, and that is what temperature really sets.