Golden-set noise: the wobble in the score
From Chapter 43 of Large Language Models from the Ground Up. A model has some fixed, hidden true skill — say it answers each eval question right 78% of the time. Score it on a small golden set and you don't measure 78%; you measure 78% plus a coin-flip wobble. This lab makes that wobble visible, and shows how it quietly reorders leaderboards.
Run the same model on the same golden set, over and over
The model's true skill is fixed at 78%. Each Run eval scores it on n questions — every question a fresh coin weighted 78% heads. Watch the measured score scatter inside the shaded band 78% ± σ, where σ = √(p(1−p)/n). Shrink n and the band explodes.
Two models, 2 points apart — how often does the worse one win?
Model A truly scores 78%; model B truly scores 76%. A is genuinely better. But on a golden set of size n, both scores wobble — and sometimes B's lucky run beats A's unlucky one. The curve is the simulated chance the worse model tops the eval, at each n.
The book is blunt about the noise floor: "at an 80% pass rate on n = 30, one standard error is about two questions (√(30 × 0.8 × 0.2) ≈ 2.2), and sampling temperature alone can move a run by 1–2 — so treat a 3-question move as a lead to investigate, and re-run before believing it." That is exactly the band you're watching: at n = 30, σ is about ±7–8 percentage points, so two runs of the same model can differ by 15 points and mean nothing. This is why "thirty is deliberately small" — big enough that a change of 3–4 answers is signal, small enough that you'll actually read the failures. And it's why the second chart matters: at small n the truly-worse model wins a real fraction of the time, so small golden sets reorder leaderboards. Grow n and the band tightens as √n, the upset rate falls — but you never buy certainty, only sharper odds. Keep your own golden set; trust it above any headline, and never believe a one-run move without re-running.