144 citations · 1.1k across the 52 of their papers we have counts for
12 papers · 2 filters
Hire-MLP: Vision MLP via Hierarchical Rearrangement
Jianyuan Guo, Yehui Tang, Kai Han +5
Previous vision MLPs such as MLP-Mixer and ResMLP accept linearly flattened image patches as input, making them inflexible for different input sizes and hard to capture spatial inf…
ReSSL: Relational Self-Supervised Learning with Weak Augmentation
Mingkai Zheng, Shan You, Fei Wang +4
Self-supervised Learning (SSL) including the mainstream contrastive learning has achieved great success in learning visual representations without data annotations. However, most o…
Augmented Shortcuts for Vision Transformers
Yehui Tang, Kai Han, Chang Xu +4
Transformer models have achieved great progress on computer vision tasks recently. The rapid development of vision transformers is mainly contributed by their high representation a…
Universal Adder Neural Networks
Hanting Chen, Yunhe Wang, Chang Xu +3
Compared with cheap addition operation, multiplication operation is of much higher computation complexity. The widely-used convolutions in deep neural networks are exactly cross-co…
K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernets
Xiu Su, Shan You, Mingkai Zheng +4
In one-shot weight sharing for NAS, the weights of each operation (at each layer) are supposed to be identical for all architectures (paths) in the supernet. However, this rules ou…
Patch Slimming for Efficient Vision Transformers
Yehui Tang, Kai Han, Yunhe Wang +4
This paper studies the efficiency problem for visual transformers by excavating redundant calculation in given networks. The recent transformer architecture has demonstrated its ef…
BCNet: Searching for Network Width with Bilaterally Coupled Network
Xiu Su, Shan You, Fei Wang +3
Searching for a more compact network width recently serves as an effective way of channel pruning for the deployment of convolutional neural networks (CNNs) under hardware constrai…
Distilling Object Detectors via Decoupled Features
Jianyuan Guo, Kai Han, Yunhe Wang +4
Knowledge distillation is a widely used paradigm for inheriting information from a complicated teacher network to a compact student network and maintaining the strong performance.…
Prioritized Architecture Sampling with Monto-Carlo Tree Search
Xiu Su, Tao Huang, Yanxi Li +5
One-shot neural architecture search (NAS) methods significantly reduce the search cost by considering the whole search space as one network, which only needs to be trained once. Ho…
Manifold Regularized Dynamic Network Pruning
Yehui Tang, Yunhe Wang, Yixing Xu +4
Neural network pruning is an essential approach for reducing the computational complexity of deep models so that they can be well deployed on resource-limited devices. Compared wit…
Learning Frequency Domain Approximation for Binary Neural Networks
Yixing Xu, Kai Han, Chang Xu +3
Binary neural networks (BNNs) represent original full-precision weights and activations into 1-bit with sign function. Since the gradient of the conventional sign function is almos…
Locally Free Weight Sharing for Network Width Search
Xiu Su, Shan You, Tao Huang +4
Searching for network width is an effective way to slim deep neural networks with hardware budgets. With this aim, a one-shot supernet is usually leveraged as a performance evaluat…