The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantization

Chao Li, Shigeng Wang, Anbang Yao

Post-training quantization (PTQ) methods typically use sequential quantization that partitions a pre-trained LLM into a series of units (e.g., transformer blocks), with one unit qu…