44 citations · 95 across the 9 of their papers we have counts for
5 papers · 1 filter
Exploring the Potential of Flexible 8-bit Format: Design and Algorithm
Zhuoyi Zhang, Yunchen Zhang, Gonglei Shi +6
Neural network quantization is widely used to reduce model inference complexity in real-world deployments. However, traditional integer quantization suffers from accuracy degradati…
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
Jing Liu, Ruihao Gong, Xiuying Wei +3
Large Language Models (LLMs) excel in NLP, but their demands hinder their widespread deployment. While Quantization-Aware Training (QAT) offers a solution, its extensive training c…
Lossy and Lossless (L) Post-training Model Size Compression
Yumeng Shi, Shihao Bai, Xiuying Wei +2
Deep neural networks have delivered remarkable performance and have been widely used in various visual tasks. However, their huge size causes significant inconvenience for transmis…
Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
Xiuying Wei, Yunchen Zhang, Yuhang Li +4
Post-training quantization~(PTQ) of transformer language models faces significant challenges due to the existence of detrimental outliers in activations. We observe that these outl…
Equiangular Basis Vectors
Yang Shen, Xuhao Sun, Xiu-Shen Wei
We propose Equiangular Basis Vectors (EBVs) for classification tasks. In deep neural networks, models usually end with a k-way fully connected layer with softmax to handle differen…