MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization

Yangjia Hu, Haodong Wang, Zicong Hong +8

4-bit quantization significantly reduces the memory footprint and accelerates the inference of large language models (LLMs). However, its limited bit-width representation struggles…