Any Orthogonal Transform Will Do: Range Reduction Explains Rotation-Based Quantization for Super-Resolution

Dorsa Zeinali, Hailing Wang, Yitian Zhang +1

Rotation-based quantization -- multiplying weights and activations by a Hadamard matrix before quantizing -- is now standard, but why it works is explained informally, via incohere…