726 citations · 4.8k across the 57 of their papers we have counts for
10 papers · 1 filter
VOLO: Vision Outlooker for Visual Recognition
Li Yuan, Qibin Hou, Zihang Jiang +2
Visual recognition has been dominated by convolutional neural networks (CNNs) for years. Though recently the prevailing vision transformers (ViTs) have shown great potential of sel…
LV-BERT: Exploiting Layer Variety for BERT
Weihao Yu, Zihang Jiang, Fei Chen +2
Modern pre-trained language models are mostly built upon backbones stacking self-attention and feed-forward layers in an interleaved order. In this paper, beyond this stereotyped l…
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition
Qibin Hou, Zihang Jiang, Li Yuan +3
In this paper, we present Vision Permutator, a conceptually simple and data efficient MLP-like architecture for visual recognition. By realizing the importance of the positional in…
Refiner: Refining Self-attention for Vision Transformers
Daquan Zhou, Yujun Shi, Bingyi Kang +6
Vision Transformers (ViTs) have shown competitive accuracy in image classification tasks compared with CNNs. Yet, they generally require much more data for model pre-training. Most…
DeepViT: Towards Deeper Vision Transformer
Daquan Zhou, Bingyi Kang, Xiaojie Jin +5
Vision transformers (ViTs) have been successfully applied in image classification tasks recently. In this paper, we show that, unlike convolution neural networks (CNNs)that can be…
All Tokens Matter: Token Labeling for Training Better Vision Transformers
Zihang Jiang, Qibin Hou, Li Yuan +5
In this paper, we present token labeling -- a new training objective for training high-performance vision transformers (ViTs). Different from the standard training objective of ViT…
AutoSpace: Neural Architecture Search with Less Human Interference
Daquan Zhou, Xiaojie Jin, Xiaochen Lian +4
Current neural architecture search (NAS) algorithms still require expert knowledge and effort to design a search space for network construction. In this paper, we consider automati…
Coordinate Attention for Efficient Mobile Network Design
Qibin Hou, Daquan Zhou, Jiashi Feng
Recent studies on mobile network design have demonstrated the remarkable effectiveness of channel attention (e.g., the Squeeze-and-Excitation attention) for lifting model performan…
FakeMix Augmentation Improves Transparent Object Detection
Yang Cao, Zhengqiang Zhang, Enze Xie +4
Detecting transparent objects in natural scenes is challenging due to the low contrast in texture, brightness and colors. Recent deep-learning-based works reveal that it is effecti…
Localization Distillation for Dense Object Detection
Zhaohui Zheng, Rongguang Ye, Ping Wang +4
Knowledge distillation (KD) has witnessed its powerful capability in learning compact models in object detection. Previous KD methods for object detection mostly focus on imitating…