Interactive labs for the whole book
Hands-on animations for the machinery in Large Language Models from the Ground Up — the parts no static figure can carry. Twenty-four labs now span the whole book, from the fundamentals of Part I to the frontier scale of Part VIII. Each lab is a playable companion to a specific chapter. Start anywhere.
The vocabulary seesaw
Chop the same sentence four ways — from single characters to whole words — and watch the seesaw tip: a tiny menu means long sequences, a huge menu means short sequences but a fortune spent on rare words.
Open lab → Ch 3The map of meaning
A training-progress slider morphs 39 everyday words from random noise into semantic neighbourhoods — hover any word to see its 3 nearest neighbours go from garbage to sensible.
Open lab → Ch 9Learning-rate playground
Roll a ball down the book's own loss valley L(w) = (w−3)². Dial the learning rate from crawl to explosion and watch the exact update either glide to the floor, zig-zag, bounce forever, or diverge.
Open lab → Ch 10The grading loop
See how a next-word guess is graded: the model puts probability on the true word and the loss is −ln(p). Toggle between hedging and confidently-wrong and watch the penalty explode.
Open lab → Ch 18 / 27BPE merge machine
Step through Byte-Pair Encoding: adjacent-pair counts recompute live, the winning pair fuses into a colored token, and the vocabulary grows.
Open lab → Ch 25The five moves
Walk the training heartbeat one move at a time — sample, forward, zero, backward, step — then let it run and watch the loss fall from 4.17 toward the book's real 1.38.
Open lab → Ch 25Overfitting lab
Train a model with sliders for size and data. Watch train and validation loss open like scissors when a big model memorizes small data — and mark where to stop.
Open lab → Ch 25Attention learns to look
Scrub 3,000 training steps of the book's GPT and watch sixteen attention heads sharpen from uniform stares into line-start, word-start, and punctuation specialists as the loss falls.
Open lab → Ch 27Watch it speak
The book's actual 825K-parameter GPT, int8-quantized and generating live in your browser — prompt it, drag temperature, top-k and top-p, and watch every character get rolled.
Open lab → Ch 29KV-cache vs naive
Generate a sequence both ways. Watch the naive model fill a quadratic work triangle while the cache adds one row — and read the wasted-work factor.
Open lab → Ch 30RoPE clock
Drag two token positions and watch the attention score depend only on their difference — rotary position encoding as spinning clock hands.
Open lab → Ch 35Reward hacking
Crank the optimization pressure and watch the reward model's score keep climbing while true quality peaks and collapses into gibberish — then leash it with a KL penalty.
Open lab → Ch 38GRPO: the group is the baseline
Sample a group of answers to one math problem, watch the checker score each, and see GRPO turn the group's own mean into "par" — no value network required.
Open lab → Ch 38Self-consistency voting
Sample many reasoning chains for one word problem, extract each final answer, and take a majority vote — watch accuracy climb as wrong slips scatter and the truth repeats.
Open lab → Ch 39Quantizer playground
Snap bell-curve weights to 8-bit or 4-bit rungs. Toggle per-group scaling and NF4, inject an outlier, and watch the real error move.
Open lab → Ch 40Continuous batching sim
Bus-seats vs taxi: see how continuous batching keeps the GPU full as requests arrive and finish, and track live utilization.
Open lab → Ch 40Speculative decoding
A cheap draft model guesses k tokens, the big target verifies all of them in one pass, and the agreeing prefix is kept — animate the accept/reject and read the speedup.
Open lab → Ch 41RAG retrieval playground
Type a question, watch it embed, score documents, pick the top-k, and land in a grounded prompt — with a hash-vs-learned embedder toggle.
Open lab → Ch 42Prompt injection
An email assistant summarizes your inbox — until one email hides an order. Watch it leak your data, then flip on the book's guardrails and stop the attack.
Open lab → Ch 43Golden-set noise
A model with fixed 78% true skill, scored again and again on a small golden set — watch the measured score wobble inside ±σ and see how often the worse of two models wins.
Open lab → Ch 43Elo arena
Stream anonymous Chatbot-Arena battles between models with hidden true strengths, and watch Elo ratings hunt down the real ranking — jumpy at first, locked-in after hundreds of votes.
Open lab → Ch 45Chinchilla allocator
Reallocate a fixed FLOPs budget between model size and tokens under C = 6·N·D, and watch the loss trace a U-shaped valley whose floor lands on Chinchilla's 20 tokens per parameter.
Open lab → Ch 46Pipeline bubbles
Watch micro-batches flow through a GPipe pipeline as a Gantt chart, with hatched "bubble" cells shrinking as you add micro-batches — and the counted idle fraction checked live against (S−1)/(M+S−1).
Open lab → Ch 463D parallelism explorer
Toggle data, tensor, and pipeline parallelism across 16 GPU tiles, colour-coded by shard, and watch the per-GPU memory and network traffic change as the DP×TP×PP degrees auto-factor to 16.
Open lab →These 24 labs span the whole book — from the Part I fundamentals through the frontier scale of Part VIII. If you're new to tokens, embeddings, attention, and sampling, the 18-lesson web course remains the gentler on-ramp.