PRQuant: Permutation Residual Quantization for Low-Overhead Inference

Peiran Wang, Anqi Wang, Jiaying Zhao +10

Low-bit quantization of linear layers is often dominated by a small number of outlier channels. Existing smoothing, rotation, and residual-based methods can mitigate this issue, bu…