PRQuant: Permutation Residual Quantization for Low-Overhead Inference

Peiran Wang, Anqi Wang, Jiaying Zhao +9

Accuracy of Low-bit quantization of linear layers is often dominated by a small number of outliers. Although existing methods, such as smoothing, rotation, or residual-based approa…