0.2 Probability, Information & Optimization

Learn

A language model is a probability distribution

Strip away everything else and a language model is one function: given the tokens so far, produce a probability for every possible next token. GPT-2 has 50,257 tokens in its vocabulary, so every forward pass ends with 50,257 numbers that are all positive and sum to 1.

The model doesn't produce probabilities directly. It produces logits — unbounded real-valued scores, one per token — and the softmax function converts them:

p_i = \frac{e^{z_i / T}}{\sum_j e^{z_j / T}}

Read it term by term: exponentiate each logit $z_i$ (making everything positive and amplifying differences), then divide by the total (making them sum to 1). The temperature $T$ divides the logits first: small $T$ exaggerates gaps between logits, large $T$ washes them out.

Key idea

Logits live on an additive scale; probabilities on a multiplicative one. Adding a constant to every logit changes nothing — only differences between logits matter. This is why interpretability work talks about “logit differences” rather than raw scores.

entropy

The expected surprise of a distribution,

H(p) = -\sum_i p_i \log_2 p_i

bits. Uniform over 5 tokens →

\log_2 5 \approx 2.32

bits. All mass on one token → 0 bits. It measures how uncertain the model is.

Training needs a score for “how wrong was that prediction?”. The universal choice is cross-entropy loss: the negative log-probability the model assigned to the token that actually came next, $\mathcal{L} = -\log p_{\text{correct}}$ . Assign the true next token 100% → loss 0. Assign it 1% → loss $-\log(0.01) \approx 4.6$ nats. Loss is literally surprise: rare-under-your-model events hurt in proportion to how confidently you ruled them out.

Reading loss numbers

Frontier-model pretraining losses land in the ballpark of ~1 nat per token. Perplexity is just

e^{\mathcal{L}}

— “the model is as confused as if it were choosing uniformly among

k

options.” A loss of 1 nat ≈ perplexity 2.7.

Learn

Training is rolling downhill

A model is a function with parameters (weights). Cross-entropy gives a single number saying how badly the current parameters predict the training data. Put those together and training becomes an optimization problem: find weights that make the loss small.

Gradient descent is the entire trick. The gradient $\nabla_\theta \mathcal{L}$ points in the direction of steepest loss increase, so step the other way:

\theta \leftarrow \theta - \eta \, \nabla_\theta \mathcal{L}

with learning rate $\eta$ controlling step size. Backpropagation is not a separate algorithm — it is the chain rule from calculus, applied efficiently through the network's computation graph so that one backward pass yields the gradient for every one of the billions of parameters at once.

The loss landscape picture: parameters define a surface; training rolls a ball toward low ground. Real landscapes have billions of dimensions — low-dimensional intuition mostly transfers, but not always.

Key idea

Everything a transformer “knows” got there because it reduced next-token surprise on training text. There is no other channel. When later modules find circuits for grammar, world knowledge, or deception-relevant features, each one exists because it paid rent in loss.

Careful

The learning rate is a genuine tradeoff, not a nuisance: too small and you crawl (or get stuck in a poor local valley); too large and you bounce past minima or diverge. You'll feel this directly in the widget below — and again in Module 1.4 when loss curves and schedules show up for real.

Explore

Feel it: softmax & descent

Two toys. First: the exact softmax computation that ends every LLM forward pass — drive the temperature to the extremes and watch entropy respond. Second: gradient descent on a bumpy 1-D loss — your job is to find a learning rate that escapes the local minimum but still converges.

Softmax & temperature

Drag the logit sliders and the temperature. Watch probability mass concentrate (T→0) or flatten (T→∞).

Temperature T1.00

logit(“cat”)2.1

logit(“dog”)1.7

logit(“car”)0.3

logit(“the”)-0.4

logit(“pizza”)-1.2

Entropy of this distribution: 1.63 bits (max 2.32 bits when uniform). Sampling at T=0 always picks cat.

Gradient descent on a bumpy loss

Step the ball downhill. Small learning rate: stuck in the local minimum. Large: it overshoots and bounces. Find a rate that escapes the first valley and settles in the global one.

Learning rate η0.30

Position x = -4.60, loss 2.686, gradient 1.769, steps 0.

Things to try: (1) Set $T$ = 0.05 — this is what greedy decoding feels like. (2) Make two logits equal and watch their probabilities lock together at any temperature. (3) In the descent toy, find the critical learning rate where behavior flips from converging to bouncing forever.

Practice

Problem set

Pencil problems first — they're quick and they make the quiz easy. The code problem is the classic rite of passage; budget an hour.

1.Softmax by handpencil & paper

Logits over a 4-token vocab: $z = (2, 1, 0, -1)$ . Compute the softmax probabilities at $T=1$ (two decimal places is fine). Then, without recomputing from scratch, say what happens to the probabilities if every logit has 10 added to it.

2.Loss as surprisepencil & paper

A model assigns the true next token probability 0.5 at position A and 0.001 at position B. What is the cross-entropy loss (in nats) at each position? How many positions like A does it take to “pay for” one position like B in the average loss?

3.The softmax gradient (guided)pencil & paper

For loss $\mathcal{L} = -\log p_c$ where $p = \mathrm{softmax}(z)$ and $c$ is the correct token, show that $\partial \mathcal{L} / \partial z_i = p_i - \mathbf{1}[i = c]$ . Interpret the result in one sentence.

4.Backprop in ~40 linescode

Follow Karpathy's micrograd video (linked in Go deeper) and build a tiny autograd engine: a Value class with +, *, tanh, and a backward() that applies the chain rule through the graph. Train a 2-layer net on XOR.

Success check: your gradients match finite differences to ~1e-4, and the XOR net reaches loss < 0.05.

0 of 4 problems marked done

Check