Identifying Sensitive Weights via Post-quantization Integral

Yuezhou Hu, Weiyu Huang, Zichen Liang +4

Serving Large Language Models (LLMs) is costly. However, post-training weight quantization can address this problem by both compressing their sizes for limited memory and saving ba…