JARQ: Joint Alternating Refinement for Quantization

Xinyu Wang, Sicheng Lyu, Xiao-Wen Chang

Group-wise post-training quantizers for large language models round weights onto a grid that is not refit to the resulting integer codes. We show that this leaves accuracy on the t…