Any Orthogonal Transform Will Do: Range Reduction Explains Rotation-Based Quantization for Super-Resolution
arXiv:2601.21069
Abstract
Rotation-based quantization --- multiplying weights and activations by a Hadamard matrix before quantizing --- is now standard, but why it works is explained informally, via incoherence, the central limit theorem, or kurtosis, and rarely tested directly. We ask which property of the post-rotation distribution drives the gain, using paired hypothesis tests on the weights and activations of an image super-resolution transformer. Three candidate properties separate cleanly. \textbf{Range reduction} replicates under every orthogonal transform we tried --- Sylvester Hadamard, QuIP\#-Paley, DCT-II, and random orthogonal matrices --- and transfers to a state-space backbone and to CNN weights. \textbf{Improved normality} also replicates. The \textbf{increased mass near zero}, by contrast, is significant only under the zero-padded Sylvester construction and disappears under unpadded transforms, so we attribute it to the padding, not the transform. The end task follows the same pattern: over a grid of three scale factors, three bitwidths and five benchmarks, rotation improves PSNR in all configurations against an otherwise identical un-rotated quantizer at the same nominal bitwidth (the rotated arm carries bits/parameter of Sylvester padding at 4 bits and stays ahead in every cell when charged for it), while substituting DCT-II or a random orthogonal matrix costs only --\,dB, less than the padding the Hadamard is charged for. The choice of kernel is therefore an implementation decision: the Hadamard's advantage is its entries and butterflies. We package the recipe as CompSRT, which improves on 2DQuant in of configurations at equal nominal bitwidth --- charged for its padding that edge holds at 3 and 2 bits but not at 4. CondiQuant remains ahead at every operating point by --\,dB, which we report.