a crash course in ~100 hours
Understand what's happening inside the model.
Transformer fundamentals → how LLMs are actually trained → mechanistic interpretability → safety, steering, and editing. Visual-first lessons, interactive toys, problem sets, and the real literature — from Attention Is All You Need to the 2026 Transformer Circuits papers on emotions and the global workspace.
Part 0Foundations
The minimum viable math: geometry, probability, optimization.
Part 1Transformer Fundamentals
Tokens, attention, the residual stream, and how training shapes it all.
1.1~2h
Tokens & Embeddings
How text becomes vectors: BPE, tokenizer pathologies, and the geometry of embedding space.
1.2~3h
Attention, Fully Understood
Queries, keys, and values as soft lookup — the mechanism that routes information between tokens.
1.3~3h
The Full Block & the Residual Stream
MLPs, LayerNorm, RoPE — and the residual stream as the shared memory everything reads and writes.
1.4~2.5h
Training Dynamics & Scaling
Next-token prediction, scaling laws, emergence debates, and grokking.
Part 2How LLMs Are Actually Made
Base models → SFT → RLHF → RLVR, plus inference and reliability.
2.1~2h
Pretraining & Base Models
What a base model is — a simulator of text, not an assistant — and the data it eats.
2.2~2.5h
Supervised Fine-Tuning & PEFT
Instruction tuning, chat templates, LoRA's low-rank insight, and catastrophic forgetting.
2.3~3h
RLHF, Reward Models & DPO
Preference learning, the KL leash, DPO's shortcut — and reward hacking as the central failure mode.
2.4~2.5h
RLVR & Reasoning Models
Verifiable rewards, DeepSeek-R1, test-time compute, and whether chains of thought tell the truth.
2.5~2.5h
Inference, Performance & Reliability
Sampling, KV caches, quantization, hallucination, and calibration — making models fast and trustworthy.
Part 3Mechanistic Interpretability: The Core
Features, circuits, superposition, SAEs, and causal methods.
3.1~2.5h
The Interp Mindset & Observational Tools
Features, circuits, and the discipline of causal evidence — plus logit lens and probes.
3.2~3.5h
A Mathematical Framework & Induction Heads
QK and OV circuits, head composition, and the induction heads behind in-context learning.
3.3~4h
Superposition: Toy Models
Why models cram more features than dimensions into their activations — the field's central obstacle.
3.4~3.5h
Sparse Autoencoders & Dictionary Learning
Decomposing superposition into monosemantic features — from Towards to Scaling Monosemanticity.
3.5~4h
Causal Methods: Patching & Circuit Discovery
Ablation, activation patching, attribution patching — and the IOI circuit as the worked example.
Part 4Frontier Interpretability
Circuit tracing, functional emotions, and the global workspace (2025–2026).
4.1~4h
Circuit Tracing & the Biology of LLMs
Transcoders, attribution graphs, and what they revealed: planning, shared circuits, and why models hallucinate.
4.2~3h
Functional Emotions
Emotion concept vectors in Claude — and how they causally mediate blackmail, reward hacking, and sycophancy.
4.3~3h
The Global Workspace & Introspection
The J-lens, verbalizable representations, ignition — and when to trust a model's self-reports.
Part 5Safety, Steering & Editing
Applying interpretability: behavior control, weight editing, and the safety landscape.
5.1~3h
Steering Behavior & Making Models Enjoyable
Activation addition, persona vectors, Golden Gate Claude — behavior control without retraining.
5.2~3h
Editing Weights & Learning on the Fly
ROME, MEMIT, test-time training — what it takes to change what a model knows.
5.3~3.5h
The AI Safety Landscape
Sleeper agents, alignment faking, auditing games — and where interpretability fits in.