144 citations · 1.1k across the 52 of their papers we have counts for
37 papers · 1 filter
Dual Focal Loss for Calibration
Linwei Tao, Minjing Dong, Chang Xu
The use of deep neural networks in real-world applications require well-calibrated networks with confidence scores that accurately reflect the actual probability. However, it has b…
Hire-MLP: Vision MLP via Hierarchical Rearrangement
Jianyuan Guo, Yehui Tang, Kai Han +5
Previous vision MLPs such as MLP-Mixer and ResMLP accept linearly flattened image patches as input, making them inflexible for different input sizes and hard to capture spatial inf…
ReSSL: Relational Self-Supervised Learning with Weak Augmentation
Mingkai Zheng, Shan You, Fei Wang +4
Self-supervised Learning (SSL) including the mainstream contrastive learning has achieved great success in learning visual representations without data annotations. However, most o…
Augmented Shortcuts for Vision Transformers
Yehui Tang, Kai Han, Chang Xu +4
Transformer models have achieved great progress on computer vision tasks recently. The rapid development of vision transformers is mainly contributed by their high representation a…
Universal Adder Neural Networks
Hanting Chen, Yunhe Wang, Chang Xu +3
Compared with cheap addition operation, multiplication operation is of much higher computation complexity. The widely-used convolutions in deep neural networks are exactly cross-co…
K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernets
Xiu Su, Shan You, Mingkai Zheng +4
In one-shot weight sharing for NAS, the weights of each operation (at each layer) are supposed to be identical for all architectures (paths) in the supernet. However, this rules ou…
Patch Slimming for Efficient Vision Transformers
Yehui Tang, Kai Han, Yunhe Wang +4
This paper studies the efficiency problem for visual transformers by excavating redundant calculation in given networks. The recent transformer architecture has demonstrated its ef…
BCNet: Searching for Network Width with Bilaterally Coupled Network
Xiu Su, Shan You, Fei Wang +3
Searching for a more compact network width recently serves as an effective way of channel pruning for the deployment of convolutional neural networks (CNNs) under hardware constrai…
Distilling Object Detectors via Decoupled Features
Jianyuan Guo, Kai Han, Yunhe Wang +4
Knowledge distillation is a widely used paradigm for inheriting information from a complicated teacher network to a compact student network and maintaining the strong performance.…
Prioritized Architecture Sampling with Monto-Carlo Tree Search
Xiu Su, Tao Huang, Yanxi Li +5
One-shot neural architecture search (NAS) methods significantly reduce the search cost by considering the whole search space as one network, which only needs to be trained once. Ho…
Manifold Regularized Dynamic Network Pruning
Yehui Tang, Yunhe Wang, Yixing Xu +4
Neural network pruning is an essential approach for reducing the computational complexity of deep models so that they can be well deployed on resource-limited devices. Compared wit…
Learning Frequency Domain Approximation for Binary Neural Networks
Yixing Xu, Kai Han, Chang Xu +3
Binary neural networks (BNNs) represent original full-precision weights and activations into 1-bit with sign function. Since the gradient of the conventional sign function is almos…
Locally Free Weight Sharing for Network Width Search
Xiu Su, Shan You, Tao Huang +4
Searching for network width is an effective way to slim deep neural networks with hardware budgets. With this aim, a one-shot supernet is usually leveraged as a performance evaluat…
PTN: A Poisson Transfer Network for Semi-supervised Few-shot Learning
Huaxi Huang, Junjie Zhang, Jian Zhang +2
The predicament in semi-supervised few-shot learning (SSFSL) is to maximize the value of the extra unlabeled data to boost the few-shot learner. In this paper, we propose a Poisson…
Pre-Trained Image Processing Transformer
Hanting Chen, Yunhe Wang, Tianyu Guo +7
As the computing power of modern hardware is increasing strongly, pre-trained deep learning models (e.g., BERT, GPT-3) learned on large-scale datasets have shown their effectivenes…
Data Agnostic Filter Gating for Efficient Deep Networks
Xiu Su, Shan You, Tao Huang +5
To deploy a well-trained CNN model on low-end computation edge devices, it is usually supposed to compress or prune the model under certain computation budget (e.g., FLOPs). Curren…
Kernel Based Progressive Distillation for Adder Neural Networks
Yixing Xu, Chang Xu, Xinghao Chen +3
Adder Neural Networks (ANNs) which only contain additions bring us a new way of developing deep neural networks with low energy consumption. Unfortunately, there is an accuracy dro…
Training Binary Neural Networks through Learning with Noisy Supervision
Kai Han, Yunhe Wang, Yixing Xu +3
This paper formalizes the binarization operations over neural networks from a learning perspective. In contrast to classical hand crafted rules (\eg hard thresholding) to binarize…
SCOP: Scientific Control for Reliable Neural Network Pruning
Yehui Tang, Yunhe Wang, Yixing Xu +4
This paper proposes a reliable neural network pruning algorithm by setting up a scientific control. Existing pruning methods have developed various hypotheses to approximate the im…
Open-Set Hypothesis Transfer with Semantic Consistency
Zeyu Feng, Chang Xu, Dacheng Tao
Unsupervised open-set domain adaptation (UODA) is a realistic problem where unlabeled target data contain unknown classes. Prior methods rely on the coexistence of both source and…
Searching for Low-Bit Weights in Quantized Neural Networks
Zhaohui Yang, Yunhe Wang, Kai Han +4
Quantized neural networks with low-bit weights and activations are attractive for developing AI accelerators. However, the quantization functions used in most conventional quantiza…
Adversarially Robust Neural Architectures
Minjing Dong, Yanxi Li, Yunhe Wang +1
Deep Neural Networks (DNNs) are vulnerable to adversarial attacks. Existing methods are devoted to developing various robust training strategies or regularizations to update the we…
Approximated Bilinear Modules for Temporal Modeling
Xinqi Zhu, Chang Xu, Langwen Hui +2
We consider two less-emphasized temporal properties of video: 1. Temporal cues are fine-grained; 2. Temporal modeling needs reasoning. To tackle both problems at once, we exploit a…
Learning Disentangled Representations with Latent Variation Predictability
Xinqi Zhu, Chang Xu, Dacheng Tao
Latent traversal is a popular approach to visualize the disentangled latent representations. Given a bunch of variations in a single unit of the latent representation, it is expect…