553 citations · 927 across the 27 of their papers we have counts for
27 papers
InCoder-32B: Code Foundation Model for Industrial Scenarios
Jian Yang, Wei Zhang, Jiajun Wu +25
Recent code large language models have achieved remarkable progress on general programming tasks. Nevertheless, their performance degrades significantly in industrial scenarios tha…
Temporal Feature Matters: A Framework for Diffusion Model Quantization
Yushi Huang, Ruihao Gong, Xianglong Liu +4
The Diffusion models, widely used for image generation, face significant challenges related to their broad applicability due to prolonged inference times and high memory demands. E…
ME-Switch: A Memory-Efficient Expert Switching Framework for Large Language Models
Jing Liu, Ruihao Gong, Mingyang Zhang +3
LLM development involves pre-training a foundation model on massive data, followed by fine-tuning on task-specific data to create specialized experts. Serving these experts can pos…
TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models
Yushi Huang, Ruihao Gong, Jing Liu +2
The Diffusion model, a prevalent framework for image generation, encounters significant challenges in terms of broad applicability due to its extended inference times and substanti…
Exploring the Potential of Flexible 8-bit Format: Design and Algorithm
Zhuoyi Zhang, Yunchen Zhang, Gonglei Shi +6
Neural network quantization is widely used to reduce model inference complexity in real-world deployments. However, traditional integer quantization suffers from accuracy degradati…
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
Jing Liu, Ruihao Gong, Xiuying Wei +3
Large Language Models (LLMs) excel in NLP, but their demands hinder their widespread deployment. While Quantization-Aware Training (QAT) offers a solution, its extensive training c…
Lossy and Lossless (L) Post-training Model Size Compression
Yumeng Shi, Shihao Bai, Xiuying Wei +2
Deep neural networks have delivered remarkable performance and have been widely used in various visual tasks. However, their huge size causes significant inconvenience for transmis…
SysNoise: Exploring and Benchmarking Training-Deployment System Inconsistency
Yan Wang, Yuhang Li, Ruihao Gong +8
Extensive studies have shown that deep learning models are vulnerable to adversarial and natural noises, yet little is known about model robustness on noises caused by different sy…
Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
Xiuying Wei, Yunchen Zhang, Yuhang Li +4
Post-training quantization~(PTQ) of transformer language models faces significant challenges due to the existence of detrimental outliers in activations. We observe that these outl…
Exploring the Relationship between Architecture and Adversarially Robust Generalization
Aishan Liu, Shiyu Tang, Siyuan Liang +4
Adversarial training has been demonstrated to be one of the most effective remedies for defending adversarial examples, yet it often suffers from the huge robustness generalization…
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models
Xiuying Wei, Yunchen Zhang, Xiangguo Zhang +5
Transformer architecture has become the fundamental element of the widespread natural language processing~(NLP) models. With the trends of large NLP models, the increasing memory a…
QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization
Xiuying Wei, Ruihao Gong, Yuhang Li +2
Recently, post-training quantization (PTQ) has driven much attention to produce efficient neural networks without long-time retraining. Despite its low cost, current PTQ works tend…
MQBench: Towards Reproducible and Deployable Model Quantization Benchmark
Yuhang Li, Mingzhu Shen, Jian Ma +6
Model quantization has emerged as an indispensable technique to accelerate deep learning inference. While researchers continue to push the frontier of quantization algorithms, exis…
Real World Robustness from Systematic Noise
Yan Wang, Yuhang Li, Ruihao Gong
Systematic error, which is not determined by chance, often refers to the inaccuracy (involving either the observation or measurement process) inherent to a system. In this paper, w…
Distribution-sensitive Information Retention for Accurate Binary Neural Network
Haotong Qin, Xiangguo Zhang, Ruihao Gong +3
Model binarization is an effective method of compressing neural networks and accelerating their inference process. However, a significant performance gap still exists between the 1…
RobustART: Benchmarking Robustness on Architecture Design and Training Techniques
Shiyu Tang, Ruihao Gong, Yan Wang +9
Deep neural networks (DNNs) are vulnerable to adversarial noises, which motivates the benchmark of model robustness. Existing benchmarks mainly focus on evaluating defenses, but th…
A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration
Yuhang Li, Shikuang Deng, Xin Dong +2
Spiking Neural Network (SNN) has been recognized as one of the next generation of neural networks. Conventionally, SNN can be converted from a pre-trained ANN by only replacing the…
Diversifying Sample Generation for Accurate Data-Free Quantization
Xiangguo Zhang, Haotong Qin, Yifu Ding +6
Quantization has emerged as one of the most prevalent approaches to compress and accelerate neural networks. Recently, data-free quantization has been widely studied as a practical…
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
Yuhang Li, Ruihao Gong, Xu Tan +6
We study the challenging task of neural network quantization without end-to-end retraining, called Post-training Quantization (PTQ). PTQ usually requires a small subset of training…
MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing
Yuhang Li, Feng Zhu, Ruihao Gong +5
User data confidentiality protection is becoming a rising challenge in the present deep learning research. Without access to data, conventional data-driven model compression faces…
Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Search
Mingzhu Shen, Feng Liang, Ruihao Gong +6
Quantization Neural Networks (QNN) have attracted a lot of attention due to their high efficiency. To enhance the quantization accuracy, prior works mainly focus on designing advan…
Binary Neural Networks: A Survey
Haotong Qin, Ruihao Gong, Xianglong Liu +3
The binary neural network, largely saving the storage and computation, serves as a promising technique for deploying deep models on resource-limited devices. However, the binarizat…
Efficient Bitwidth Search for Practical Mixed Precision Neural Network
Yuhang Li, Wei Wang, Haoli Bai +3
Network quantization has rapidly become one of the most widely used methods to compress and accelerate deep neural networks. Recent efforts propose to quantize weights and activati…
Towards Unified INT8 Training for Convolutional Neural Network
Feng Zhu, Ruihao Gong, Fengwei Yu +5
Recently low-bit (e.g., 8-bit) network quantization has been extensively studied to accelerate the inference. Besides inference, low-bit training with quantized gradients can furth…