4.4k citations · 7.7k across the 83 of their papers we have counts for
24 papers · 1 filter
An Empirical Study of Adder Neural Networks for Object Detection
Xinghao Chen, Chang Xu, Minjing Dong +2
Adder neural networks (AdderNets) have shown impressive performance on image classification with only addition operations, which are more energy efficient than traditional convolut…
An Image Patch is a Wave: Phase-Aware Vision MLP
Yehui Tang, Kai Han, Jianyuan Guo +4
In the field of computer vision, recent works show that a pure MLP architecture mainly stacked by fully-connected layers can achieve competing performance with CNN and transformer.…
Learning Versatile Convolution Filters for Efficient Visual Recognition
Kai Han, Yunhe Wang, Chang Xu +3
This paper introduces versatile filters to construct efficient convolutional neural networks that are widely used in various visual recognition tasks. Considering the demands of ef…
Neural Architecture Dilation for Adversarial Robustness
Yanxi Li, Zhaohui Yang, Yunhe Wang +1
With the tremendous advances in the architecture and scale of convolutional neural networks (CNNs) over the past few decades, they can easily reach or even exceed the performance o…
Hire-MLP: Vision MLP via Hierarchical Rearrangement
Jianyuan Guo, Yehui Tang, Kai Han +5
Previous vision MLPs such as MLP-Mixer and ResMLP accept linearly flattened image patches as input, making them inflexible for different input sizes and hard to capture spatial inf…
Greedy Network Enlarging
Chuanjian Liu, Kai Han, An Xiao +4
Recent studies on deep convolutional neural networks present a simple paradigm of architecture design, i.e., models with more MACs typically achieve better accuracy, such as Effici…
CMT: Convolutional Neural Networks Meet Vision Transformers
Jianyuan Guo, Kai Han, Han Wu +4
Vision transformers have been successfully applied to image recognition tasks due to their ability to capture long-range dependencies within an image. However, there are still gaps…
Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation
Zhiwei Hao, Jianyuan Guo, Ding Jia +5
In the past few years, transformers have achieved promising performances on various computer vision tasks. Unfortunately, the immense inference overhead of most existing vision tra…
Augmented Shortcuts for Vision Transformers
Yehui Tang, Kai Han, Chang Xu +4
Transformer models have achieved great progress on computer vision tasks recently. The rapid development of vision transformers is mainly contributed by their high representation a…
Universal Adder Neural Networks
Hanting Chen, Yunhe Wang, Chang Xu +3
Compared with cheap addition operation, multiplication operation is of much higher computation complexity. The widely-used convolutions in deep neural networks are exactly cross-co…
Post-Training Quantization for Vision Transformer
Zhenhua Liu, Yunhe Wang, Kai Han +2
Recently, transformer has achieved remarkable performance on a variety of computer vision applications. Compared with mainstream convolutional neural networks, vision transformers…
Federated Learning with Positive and Unlabeled Data
Xinyang Lin, Hanting Chen, Yixing Xu +4
We study the problem of learning from positive and unlabeled (PU) data in the federated setting, where each client only labels a little part of their dataset due to the limitation…
Dynamic Resolution Network
Mingjian Zhu, Kai Han, Enhua Wu +4
Deep convolutional neural networks (CNNs) are often of sophisticated design with numerous learnable parameters for the accuracy reason. To alleviate the expensive costs of deployin…
Patch Slimming for Efficient Vision Transformers
Yehui Tang, Kai Han, Yunhe Wang +4
This paper studies the efficiency problem for visual transformers by excavating redundant calculation in given networks. The recent transformer architecture has demonstrated its ef…
Winograd Algorithm for AdderNet
Wenshuo Li, Hanting Chen, Mingqiang Huang +3
Adder neural network (AdderNet) is a new kind of deep model that replaces the original massive multiplications in convolutions by additions while preserving the high performance. S…
Coarse-to-Fine Searching for Efficient Generative Adversarial Networks
Jiahao Wang, Han Shu, Weihao Xia +2
This paper studies the neural architecture search (NAS) problem for developing efficient generator networks. Compared with deep models for visual recognition tasks, generative adve…
Vision Transformer Pruning
Mingjian Zhu, Yehui Tang, Kai Han
Vision transformer has achieved competitive performance on a variety of computer vision applications. However, their storage, run-time memory, and computational demands are hinderi…
Distilling Object Detectors via Decoupled Features
Jianyuan Guo, Kai Han, Yunhe Wang +4
Knowledge distillation is a widely used paradigm for inheriting information from a complicated teacher network to a compact student network and maintaining the strong performance.…
Manifold Regularized Dynamic Network Pruning
Yehui Tang, Yunhe Wang, Yixing Xu +4
Neural network pruning is an essential approach for reducing the computational complexity of deep models so that they can be well deployed on resource-limited devices. Compared wit…
Learning Frequency-aware Dynamic Network for Efficient Super-Resolution
Wenbin Xie, Dehua Song, Chang Xu +3
Deep learning based methods, especially convolutional neural networks (CNNs) have been successfully applied in the field of single image super-resolution (SISR). To obtain better f…
Learning Frequency Domain Approximation for Binary Neural Networks
Yixing Xu, Kai Han, Chang Xu +3
Binary neural networks (BNNs) represent original full-precision weights and activations into 1-bit with sign function. Since the gradient of the conventional sign function is almos…
Transformer in Transformer
Kai Han, An Xiao, Enhua Wu +3
Transformer is a new kind of neural architecture which encodes the input data as powerful features via the attention mechanism. Basically, the visual transformers first divide the…
AdderNet and its Minimalist Hardware Design for Energy-Efficient Artificial Intelligence
Yunhe Wang, Mingqiang Huang, Kai Han +4
Convolutional neural networks (CNN) have been widely used for boosting the performance of many machine intelligence tasks. However, the CNN models are usually computationally inten…
GhostSR: Learning Ghost Features for Efficient Image Super-Resolution
Ying Nie, Kai Han, Zhenhua Liu +2
Modern single image super-resolution (SISR) system based on convolutional neural networks (CNNs) achieves fancy performance while requires huge computational costs. The problem on…