167 citations · 914 across the 103 of their papers we have counts for
8 papers · 1 filter
Sharpness-aware Quantization for Deep Neural Networks
Jing Liu, Jianfei Cai, Bohan Zhuang
Network quantization is a dominant paradigm of model compression. However, the abrupt changes in quantized weights during training often lead to severe loss fluctuations and result…
Pruning Self-attentions into Convolutional Layers in Single Path
Haoyu He, Jianfei Cai, Jing Liu +4
Vision Transformers (ViTs) have achieved impressive performance over various computer vision tasks. However, modeling global correlations with multi-head self-attention (MSA) layer…
Mesa: A Memory-saving Training Framework for Transformers
Zizheng Pan, Peng Chen, Haoyu He +3
There has been an explosion of interest in designing high-performance Transformers. While Transformers have delivered significant performance improvements, training such networks i…
Rapid Elastic Architecture Search under Specialized Classes and Resource Constraints
Jing Liu, Bohan Zhuang, Mingkui Tan +4
In many real-world applications, we often need to handle various deployment scenarios, where the resource constraint and the superclass of interest corresponding to a group of clas…
Less is More: Pay Less Attention in Vision Transformers
Zizheng Pan, Bohan Zhuang, Haoyu He +2
Transformers have become one of the dominant architectures in deep learning, particularly as a powerful alternative to convolutional neural networks (CNNs) in computer vision. Howe…
End-to-end One-shot Human Parsing
Haoyu He, Bohan Zhuang, Jing Zhang +2
Previous human parsing methods are limited to parsing humans into pre-defined classes, which is inflexible for practical fashion applications that often have new fashion item class…
Scalable Vision Transformers with Hierarchical Pooling
Zizheng Pan, Bohan Zhuang, Jing Liu +2
The recently proposed Visual image Transformers (ViT) with pure attention have achieved promising performance on image recognition tasks, such as image classification. However, the…
Single-path Bit Sharing for Automatic Loss-aware Model Compression
Jing Liu, Bohan Zhuang, Peng Chen +3
Network pruning and quantization are proven to be effective ways for deep model compression. To obtain a highly compact model, most methods first perform network pruning and then c…