6 papers
First-Order Error Matters: Accurate Compensation for Quantized Large Language Models
Xingyu Zheng, Haotong Qin, Yuye Li +5
Post-training quantization (PTQ) offers an efficient approach to compressing large language models (LLMs), significantly reducing memory access and computational costs. Existing co…
A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms
Ruihao Gong, Yifu Ding, Zining Wang +7
Large language models (LLMs) have achieved remarkable advancements in natural language processing, showcasing exceptional performance across various tasks. However, the expensive m…
SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
Wei Huang, Haotong Qin, Yangdong Liu +7
Post-training quantization (PTQ) is an effective technique for compressing large language models (LLMs). However, while uniform-precision quantization is computationally efficient,…
Low-bit Model Quantization for Deep Neural Networks: A Survey
Kai Liu, Qian Zheng, Kaiwen Tao +9
With unprecedented rapid development, deep neural networks (DNNs) have deeply influenced almost all fields. However, their heavy computation costs and model sizes are usually unacc…
An Empirical Study of Qwen3 Quantization
Xingyu Zheng, Yuye Li, Haoran Chu +7
The Qwen series has emerged as a leading family of open-source Large Language Models (LLMs), demonstrating remarkable capabilities in natural language understanding tasks. With the…
An empirical study of LLaMA3 quantization: from LLMs to MLLMs
Wei Huang, Xingyu Zheng, Xudong Ma +7
The LLaMA family, a collection of foundation language models ranging from 7B to 65B parameters, has become one of the most powerful open-source large language models (LLMs) and the…