How LLMs Work
Book Labs
All labs
03 Book labs · Embeddings

The map organizes itself

From Chapter 3 of Large Language Models from the Ground Up. Every word starts life as random numbers — meaningless static. Then trillions of next-word nudges pull words used in similar ways together, until the noise becomes a map where near means similar and directions mean relationships. Drag the training slider and watch it happen.

Interactive

From noise to neighborhoods

Hover or tap a word to light up its 3 nearest neighbours, measured live at the current positions. Early in training they're nonsense; by the end they're the word's true semantic neighbours.

Training progressstep 0 — random noise
Hover a word to see its nearest neighbours — and drag the slider to watch them go from garbage to sensible.
What to notice

At progress 0, hover "king": its nearest neighbours are whatever random static happens to sit close — "pizza", "swim", nonsense, because nobody drew the map yet. Drag to 100% and the same hover returns "queen", "prince", "man": the royalty neighbourhood. Push the slider past ~80% and two parallel arrows appear — man→king and woman→queen are the same step, which is why king − man + woman ≈ queen. Nobody labelled a cluster or drew an arrow; the geometry is a fossil of usage, squeezed out because giving interchangeable words similar numbers is what wins the guessing game.

← Back to all labs