Foundations

Softmax, temperature and cross-entropy

The softmax function turns raw logits into probabilities in two moves: exponentiate, then divide by the sum. Watch a machine do it with the real numbers printed at every step, then heat it with temperature and read the loss off a gauge.

Logits
Temperature 1.00
True label
KL direction
Loading 3D view

LogitsEvery step ends with one raw score per candidate token, called a logit: here Paris scores four, Lyon two, France one and a half, and the word the minus one, and none of that is a probability yet.

Narrated with James Phoenix's AI voice.

1.00
Temperature
Paris
Label
0.89bits
Entropy
0.29bits
Cross-entropy
0.00bits
KL (T=1 ‖ heated)
0.00bits
KL (heated ‖ T=1)
1.86of 4
Perplexity

One logit two points clear of the next. The exponential turns that gap into a seven-to-one ratio, so Paris takes about four fifths of the tube. exp of the top logit is 54.6 out of a total of 66.8, which is 81.7% for Paris; the gauge reads 0.29 bits, and KL in this direction is 0.00 bits.

A language model does not emit a word. It emits a list of raw scores, one per token in its vocabulary, and those scores are called logits. They can be any real number, they do not add up to anything, and you cannot sample from them. Press play and the film opens on four of them, printed on cards in a rack: 4.0 for Paris, 2.0 for Lyon, 1.5 for France, minus 1.0 for the, the model's scores for the next token after "The capital of France is".

The softmax function is two moves, and the machine does them in order. First the bars ride into the glass chamber and each grows to the exponential of its score, with the new number printed above it: e to the 4 is about 55, e to the 2 about 7.4, and e to the minus 1 is 0.37, still positive. That is the step people skip over, and it is where the shape of the answer is decided. A gap of two logit points is not a gap of two of anything; it is a ratio of about seven to one, because the exponential turns differences into ratios.

Second, the grown bars pour into the share tube and are divided by their total. Now the segments add to exactly one and each is a probability: about 80% for Paris on the confident preset, 11% for Lyon, and so on down. The tube is the whole reason the exponential is there. Exponentials are always positive, so every token gets a share, and dividing by the sum makes the shares a distribution you can sample from.

Temperature is the dial on the side of the chamber. It divides every logit before the exponential. Below one the gaps between logits are stretched, the biggest exponential runs away from the rest and one segment takes over the tube; above one the gaps are squashed, the exponentials level out and the tube divides more evenly. The film sweeps the dial from cold to hot and back to your slider so the shares and the printed numbers can be watched changing together.

Cross-entropy is what training minimises, and against a single true label it is nothing more than minus the log of the share given to that label. The gauge on the right reads it in bits. In the film the true label's segment lights gold and the needle settles; then the label switches briefly to Lyon, the lit segment shrinks, and the needle jumps hard right. The loss never looks at the other segments. It only asks how much the model gave the right answer.

KL divergence is the one that most explanations leave vague. It compares two distributions, not a distribution and a label. The film stacks a second tube above the first, holding the heated shares, and draws a bar between them for each token: that token's share times the log of the ratio of the two shares. The bars add up to the divergence. Swap which tube does the believing and the bars and their total change, which is the whole point: KL is not a distance, and the direction matters.

Entropy is the one-number summary of how spread the tube is, in bits: zero when one segment has everything, two bits when four segments are equal. Perplexity is two to the power of that, and the last chapter redraws the tube as that many equal slices. A perplexity of 1.7 means the model is exactly as uncertain as a fair choice between 1.7 equally likely options. Four tokens is a simplification that costs nothing: a real vocabulary has around a hundred thousand, and the machine is the same, only the tube has more segments.

The maths

The softmax function with temperature
qi  =  exp(zi/T)jexp(zj/T)q_i \;=\; \frac{\exp(z_i / T)}{\sum_j \exp(z_j / T)}

The logits z are the printed scores on the cards; T is the dial on the chamber. Dividing by T before exponentiating stretches or squashes the gaps between logits. As T falls towards zero the largest exponential dwarfs the rest and one segment takes the whole tube; as T grows the exponentials level out and the tube divides evenly.

Cross-entropy loss against the true label
H(y,q)  =  iyilog2qi  =  log2qlabelH(y, q) \;=\; -\sum_i y_i \log_2 q_i \;=\; -\log_2 q_{\text{label}}

With a one-hot target y this collapses to minus the log probability the model gave the correct token. It is the training loss for every language model. On screen it is the gauge: the needle reads the log of the lit segment in the tube, near zero when the model is confidently right, hard right when it is confidently wrong, off the scale if the label gets no share at all.

KL divergence between two distributions
DKL(pq)  =  ipilog2piqi  =  H(p,q)H(p)D_{\mathrm{KL}}(p \,\|\, q) \;=\; \sum_i p_i \log_2 \frac{p_i}{q_i} \;=\; H(p, q) - H(p)

The extra bits you pay for believing q when the truth is p. Here p is the tube at temperature one and q the heated tube above it, and the bars between them are the per-token terms of the sum. Swap the direction and the terms and their total change: KL is not a distance, and which side does the believing matters.

Entropy and perplexity
H(q)  =  iqilog2qi,PPL  =  2HH(q) \;=\; -\sum_i q_i \log_2 q_i, \qquad \mathrm{PPL} \;=\; 2^{H}

Entropy is how undecided the distribution is, in bits. Perplexity is the same number expressed as an effective count of options: a flat distribution over four tokens has perplexity four. Reported language-model perplexities are two (or e) to the power of the average cross-entropy over a test set, which is why a lower number is better.

Related terms

More visualisations

Building with language models?

These explainers come out of the work. If you want the same thinking applied to your own system, that is what I do.

See how I can help