2 papers
cs.LG2025
MergeQuant: Accurate 4-bit Static Quantization of Large Language Models by Channel-wise Calibration
Jinguang Wang, Jingyu Wang, Haifeng Sun +6
Quantization has been widely used to compress and accelerate inference of large language models (LLMs). Existing methods focus on exploring the per-token dynamic calibration to ens…
cs.CL2024
OutlierTune: Efficient Channel-Wise Quantization for Large Language Models
Jinguang Wang, Yuexi Yin, Haifeng Sun +5
Quantizing the activations of large language models (LLMs) has been a significant challenge due to the presence of structured outliers. Most existing methods focus on the per-token…