553 citations · 927 across the 27 of their papers we have counts for
9 papers · 1 filter
SysNoise: Exploring and Benchmarking Training-Deployment System Inconsistency
Yan Wang, Yuhang Li, Ruihao Gong +8
Extensive studies have shown that deep learning models are vulnerable to adversarial and natural noises, yet little is known about model robustness on noises caused by different sy…
Exploring the Relationship between Architecture and Adversarially Robust Generalization
Aishan Liu, Shiyu Tang, Siyuan Liang +4
Adversarial training has been demonstrated to be one of the most effective remedies for defending adversarial examples, yet it often suffers from the huge robustness generalization…
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models
Xiuying Wei, Yunchen Zhang, Xiangguo Zhang +5
Transformer architecture has become the fundamental element of the widespread natural language processing~(NLP) models. With the trends of large NLP models, the increasing memory a…
MQBench: Towards Reproducible and Deployable Model Quantization Benchmark
Yuhang Li, Mingzhu Shen, Jian Ma +6
Model quantization has emerged as an indispensable technique to accelerate deep learning inference. While researchers continue to push the frontier of quantization algorithms, exis…
A Free Lunch From ANN: Towards Efficient, Accurate Spiking Neural Networks Calibration
Yuhang Li, Shikuang Deng, Xin Dong +2
Spiking Neural Network (SNN) has been recognized as one of the next generation of neural networks. Conventionally, SNN can be converted from a pre-trained ANN by only replacing the…
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
Yuhang Li, Ruihao Gong, Xu Tan +6
We study the challenging task of neural network quantization without end-to-end retraining, called Post-training Quantization (PTQ). PTQ usually requires a small subset of training…
MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing
Yuhang Li, Feng Zhu, Ruihao Gong +5
User data confidentiality protection is becoming a rising challenge in the present deep learning research. Without access to data, conventional data-driven model compression faces…
Efficient Bitwidth Search for Practical Mixed Precision Neural Network
Yuhang Li, Wei Wang, Haoli Bai +3
Network quantization has rapidly become one of the most widely used methods to compress and accelerate deep neural networks. Recent efforts propose to quantize weights and activati…
Towards Unified INT8 Training for Convolutional Neural Network
Feng Zhu, Ruihao Gong, Fengwei Yu +5
Recently low-bit (e.g., 8-bit) network quantization has been extensively studied to accelerate the inference. Besides inference, low-bit training with quantized gradients can furth…