144 citations · 1.1k across the 52 of their papers we have counts for
16 papers · 1 filter
Hire-MLP: Vision MLP via Hierarchical Rearrangement
Jianyuan Guo, Yehui Tang, Kai Han +5
Previous vision MLPs such as MLP-Mixer and ResMLP accept linearly flattened image patches as input, making them inflexible for different input sizes and hard to capture spatial inf…
ReSSL: Relational Self-Supervised Learning with Weak Augmentation
Mingkai Zheng, Shan You, Fei Wang +4
Self-supervised Learning (SSL) including the mainstream contrastive learning has achieved great success in learning visual representations without data annotations. However, most o…
Augmented Shortcuts for Vision Transformers
Yehui Tang, Kai Han, Chang Xu +4
Transformer models have achieved great progress on computer vision tasks recently. The rapid development of vision transformers is mainly contributed by their high representation a…
Universal Adder Neural Networks
Hanting Chen, Yunhe Wang, Chang Xu +3
Compared with cheap addition operation, multiplication operation is of much higher computation complexity. The widely-used convolutions in deep neural networks are exactly cross-co…
K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernets
Xiu Su, Shan You, Mingkai Zheng +4
In one-shot weight sharing for NAS, the weights of each operation (at each layer) are supposed to be identical for all architectures (paths) in the supernet. However, this rules ou…
Commutative Lie Group VAE for Disentanglement Learning
Xinqi Zhu, Chang Xu, Dacheng Tao
We view disentanglement learning as discovering an underlying structure that equivariantly reflects the factorized variations shown in data. Traditionally, such a structure is fixe…
Patch Slimming for Efficient Vision Transformers
Yehui Tang, Kai Han, Yunhe Wang +4
This paper studies the efficiency problem for visual transformers by excavating redundant calculation in given networks. The recent transformer architecture has demonstrated its ef…
BCNet: Searching for Network Width with Bilaterally Coupled Network
Xiu Su, Shan You, Fei Wang +3
Searching for a more compact network width recently serves as an effective way of channel pruning for the deployment of convolutional neural networks (CNNs) under hardware constrai…
Joint Distribution across Representation Space for Out-of-Distribution Detection
JingWei Xu, Siyuan Zhu, Zenan Li +1
Deep neural networks (DNNs) have become a key part of many modern software applications. After training and validating, the DNN is deployed as an irrevocable component and applied…
Distilling Object Detectors via Decoupled Features
Jianyuan Guo, Kai Han, Yunhe Wang +4
Knowledge distillation is a widely used paradigm for inheriting information from a complicated teacher network to a compact student network and maintaining the strong performance.…
Prioritized Architecture Sampling with Monto-Carlo Tree Search
Xiu Su, Tao Huang, Yanxi Li +5
One-shot neural architecture search (NAS) methods significantly reduce the search cost by considering the whole search space as one network, which only needs to be trained once. Ho…
Manifold Regularized Dynamic Network Pruning
Yehui Tang, Yunhe Wang, Yixing Xu +4
Neural network pruning is an essential approach for reducing the computational complexity of deep models so that they can be well deployed on resource-limited devices. Compared wit…
Learning Frequency-aware Dynamic Network for Efficient Super-Resolution
Wenbin Xie, Dehua Song, Chang Xu +3
Deep learning based methods, especially convolutional neural networks (CNNs) have been successfully applied in the field of single image super-resolution (SISR). To obtain better f…
Learning Frequency Domain Approximation for Binary Neural Networks
Yixing Xu, Kai Han, Chang Xu +3
Binary neural networks (BNNs) represent original full-precision weights and activations into 1-bit with sign function. Since the gradient of the conventional sign function is almos…
LocalDrop: A Hybrid Regularization for Deep Neural Networks
Ziqing Lu, Chang Xu, Bo Du +3
In neural networks, developing regularization algorithms to settle overfitting is one of the major study areas. We propose a new approach for the regularization of neural networks…
Locally Free Weight Sharing for Network Width Search
Xiu Su, Shan You, Tao Huang +4
Searching for network width is an effective way to slim deep neural networks with hardware budgets. With this aim, a one-shot supernet is usually leveraged as a performance evaluat…