The quantizer playground
From Chapter 39 of Large Language Models from the Ground Up. Quantization stores each weight as one of a small set of levels — 256 for 8-bit, just 16 for 4-bit. The arithmetic below is real: weights are quantized and de-quantized exactly as the book describes, and the error is measured, not faked.
Snap the weights to a grid
A batch of ~48 bell-curve weights. Choose the precision and scheme; watch each weight snap to its nearest rung, and read the true mean-absolute error. Then flip inject outlier and see what one big value does.
Turn on the outlier with 4-bit integer, per-tensor. One value of 8.0 stretches the single shared scale so far that the ordinary small weights all round to the same rung — zero. The information in them is gone. Now switch to per-group: the outlier only blows out its own group of 16; the other groups keep a tight scale and the small weights come back to life, and the error drops. This is precisely why 4-bit schemes are group-wise, and why NF4 (levels bunched near zero, straight from the QLoRA paper) beats plain integer for bell-shaped weights.