28 citations · 37 across the 7 of their papers we have counts for
4 papers · 1 filter
Exploring the Potential of Flexible 8-bit Format: Design and Algorithm
Zhuoyi Zhang, Yunchen Zhang, Gonglei Shi +6
Neural network quantization is widely used to reduce model inference complexity in real-world deployments. However, traditional integer quantization suffers from accuracy degradati…
TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise
Nan He, Hanyu Lai, Chenyang Zhao +12
Large Language Models (LLMs) exhibit impressive reasoning and data augmentation capabilities in various NLP tasks. However, what about small models? In this work, we propose Teache…
SysNoise: Exploring and Benchmarking Training-Deployment System Inconsistency
Yan Wang, Yuhang Li, Ruihao Gong +8
Extensive studies have shown that deep learning models are vulnerable to adversarial and natural noises, yet little is known about model robustness on noises caused by different sy…
Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
Xiuying Wei, Yunchen Zhang, Yuhang Li +4
Post-training quantization~(PTQ) of transformer language models faces significant challenges due to the existence of detrimental outliers in activations. We observe that these outl…