4.4k citations · 5.3k across the 63 of their papers we have counts for
10 papers · 1 filter
An Image Patch is a Wave: Phase-Aware Vision MLP
Yehui Tang, Kai Han, Jianyuan Guo +4
In the field of computer vision, recent works show that a pure MLP architecture mainly stacked by fully-connected layers can achieve competing performance with CNN and transformer.…
Hire-MLP: Vision MLP via Hierarchical Rearrangement
Jianyuan Guo, Yehui Tang, Kai Han +5
Previous vision MLPs such as MLP-Mixer and ResMLP accept linearly flattened image patches as input, making them inflexible for different input sizes and hard to capture spatial inf…
Homogeneous Architecture Augmentation for Neural Predictor
Yuqiao Liu, Yehui Tang, Yanan Sun
Neural Architecture Search (NAS) can automatically design well-performed architectures of Deep Neural Networks (DNNs) for the tasks at hand. However, one bottleneck of NAS is the p…
CMT: Convolutional Neural Networks Meet Vision Transformers
Jianyuan Guo, Kai Han, Han Wu +4
Vision transformers have been successfully applied to image recognition tasks due to their ability to capture long-range dependencies within an image. However, there are still gaps…
Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation
Zhiwei Hao, Jianyuan Guo, Ding Jia +5
In the past few years, transformers have achieved promising performances on various computer vision tasks. Unfortunately, the immense inference overhead of most existing vision tra…
Augmented Shortcuts for Vision Transformers
Yehui Tang, Kai Han, Chang Xu +4
Transformer models have achieved great progress on computer vision tasks recently. The rapid development of vision transformers is mainly contributed by their high representation a…
Patch Slimming for Efficient Vision Transformers
Yehui Tang, Kai Han, Yunhe Wang +4
This paper studies the efficiency problem for visual transformers by excavating redundant calculation in given networks. The recent transformer architecture has demonstrated its ef…
Vision Transformer Pruning
Mingjian Zhu, Yehui Tang, Kai Han
Vision transformer has achieved competitive performance on a variety of computer vision applications. However, their storage, run-time memory, and computational demands are hinderi…
Manifold Regularized Dynamic Network Pruning
Yehui Tang, Yunhe Wang, Yixing Xu +4
Neural network pruning is an essential approach for reducing the computational complexity of deep models so that they can be well deployed on resource-limited devices. Compared wit…
Learning Frequency Domain Approximation for Binary Neural Networks
Yixing Xu, Kai Han, Chang Xu +3
Binary neural networks (BNNs) represent original full-precision weights and activations into 1-bit with sign function. Since the gradient of the conventional sign function is almos…