📚 What am I looking at?
Error propagation
Each quantized forward pass is compared with the FP32 reference at every transformer layer. Errors compound: a perturbation in layer 1 is amplified by every subsequent matmul, LayerNorm and attention mixing step — so the curves trend upward, and INT4 (16 levels) sits far above INT8 (256).
Where outliers live
The heatmap shows per-channel max |activation| after each layer. A few fixed channels carry values 10–100× larger than the rest (bright vertical stripes). They emerge in early-middle layers and persist — they ride the residual stream. With per-tensor activation quantization, one outlier channel sets the scale for all channels, crushing the rest into a few integer levels. That's why W8A8 naive is so much worse than INT8 weight-only.
Why SmoothQuant targets what it targets
Outliers live in activations, in specific channels;
weights are flat and easy. SmoothQuant migrates difficulty: divide
activations by s_j = |X_j|max^α / |W_j|max^(1−α) and multiply
the matching weight columns by s_j — a mathematical no-op that
makes both tensors "medium hard" instead of one impossible + one trivial.
Compare W8A8 naive vs W8A8 + SmoothQuant: same bits, very
different error. Try α = 0.3 / 0.5 / 0.7 above.
Why AWQ targets what it targets
For weight-only INT4, the weight columns that multiply large activations (the same outlier channels!) contribute most of the output error. AWQ scales those salient columns up before rounding — shrinking their relative rounding error — and folds the inverse scale into the activation path. Compare INT4 g128 vs INT4 g128 + AWQ-style at α = 0.3. Push α too high and the boost itself inflates the group scales and error grows again — precisely why real AWQ searches for the best per-layer scale instead of fixing α. (INT4 here is group-wise, g = 128, as deployed in practice.)
Kurtosis
Excess kurtosis of hidden states per layer (0 = Gaussian). Values in the thousands are the statistical fingerprint of extreme outlier channels.
All quantization is simulated ("fake quant"): round-to-nearest onto a symmetric integer grid in float32 — exactly the error a real integer kernel introduces. Source: GitHub.