4.4k citations · 7.7k across the 83 of their papers we have counts for
22 papers · 1 filter
A Survey on Visual Transformer
Kai Han, Yunhe Wang, Hanting Chen +10
Transformer, first applied to the field of natural language processing, is a type of deep neural network mainly based on the self-attention mechanism. Thanks to its strong represen…
Dynamic Feature Pyramid Networks for Object Detection
Mingjian Zhu, Kai Han, Changbin Yu +1
Feature pyramid network (FPN) is a critical component in modern object detection frameworks. The performance gain in most of the existing FPN variants is mainly attributed to the i…
Pre-Trained Image Processing Transformer
Hanting Chen, Yunhe Wang, Tianyu Guo +7
As the computing power of modern hardware is increasing strongly, pre-trained deep learning models (e.g., BERT, GPT-3) learned on large-scale datasets have shown their effectivenes…
Kernel Based Progressive Distillation for Adder Neural Networks
Yixing Xu, Chang Xu, Xinghao Chen +3
Adder Neural Networks (ANNs) which only contain additions bring us a new way of developing deep neural networks with low energy consumption. Unfortunately, there is an accuracy dro…
Training Binary Neural Networks through Learning with Noisy Supervision
Kai Han, Yunhe Wang, Yixing Xu +3
This paper formalizes the binarization operations over neural networks from a learning perspective. In contrast to classical hand crafted rules (\eg hard thresholding) to binarize…
Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets
Kai Han, Yunhe Wang, Qiulin Zhang +3
To obtain excellent deep neural architectures, a series of techniques are carefully designed in EfficientNets. The giant formula for simultaneously enlarging the resolution, depth…
SCOP: Scientific Control for Reliable Neural Network Pruning
Yehui Tang, Yunhe Wang, Yixing Xu +4
This paper proposes a reliable neural network pruning algorithm by setting up a scientific control. Existing pruning methods have developed various hypotheses to approximate the im…
Searching for Low-Bit Weights in Quantized Neural Networks
Zhaohui Yang, Yunhe Wang, Kai Han +4
Quantized neural networks with low-bit weights and activations are attractive for developing AI accelerators. However, the quantization functions used in most conventional quantiza…
AdderSR: Towards Energy Efficient Image Super-Resolution
Dehua Song, Yunhe Wang, Hanting Chen +3
This paper studies the single image super-resolution problem using adder neural networks (AdderNet). Compared with convolutional neural networks, AdderNet utilizing additions to ca…
Adversarially Robust Neural Architectures
Minjing Dong, Yanxi Li, Yunhe Wang +1
Deep Neural Networks (DNNs) are vulnerable to adversarial attacks. Existing methods are devoted to developing various robust training strategies or regularizations to update the we…
Optical Flow Distillation: Towards Efficient and Stable Video Style Transfer
Xinghao Chen, Yiman Zhang, Yunhe Wang +3
Video style transfer techniques inspire many exciting applications on mobile devices. However, their efficiency and stability are still far from satisfactory. To boost the transfer…
MTP: Multi-Task Pruning for Efficient Semantic Segmentation Networks
Xinghao Chen, Yiman Zhang, Yunhe Wang
This paper focuses on channel pruning for semantic segmentation networks. Previous methods to compress and accelerate deep neural networks in the classification task cannot be stra…
DC-NAS: Divide-and-Conquer Neural Architecture Search
Yunhe Wang, Yixing Xu, Dacheng Tao
Most applications demand high-performance deep neural architectures costing limited resources. Neural architecture searching is a way of automatically exploring optimal deep neural…
A Semi-Supervised Assessor of Neural Architectures
Yehui Tang, Yunhe Wang, Yixing Xu +6
Neural architecture search (NAS) aims to automatically design deep neural networks of satisfactory performance. Wherein, architecture performance predictor is critical to efficient…
HourNAS: Extremely Fast Neural Architecture Search Through an Hourglass Lens
Zhaohui Yang, Yunhe Wang, Xinghao Chen +6
Neural Architecture Search (NAS) refers to automatically design the architecture. We propose an hourglass-inspired approach (HourNAS) for this problem that is motivated by the fact…
Hit-Detector: Hierarchical Trinity Architecture Search for Object Detection
Jianyuan Guo, Kai Han, Yunhe Wang +5
Neural Architecture Search (NAS) has achieved great success in image classification task. Some recent works have managed to explore the automatic design of efficient backbone or fe…
Distilling portable Generative Adversarial Networks for Image Translation
Hanting Chen, Yunhe Wang, Han Shu +5
Despite Generative Adversarial Networks (GANs) have been widely used in various image-to-image translation tasks, they can be hardly applied on mobile devices due to their heavy co…
Automatically Searching for U-Net Image Translator Architecture
Han Shu, Yunhe Wang
Image translators have been successfully applied to many important low level image processing tasks. However, classical network architecture of image translator like U-Net, is borr…
Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks
Yehui Tang, Yunhe Wang, Yixing Xu +4
Deep neural networks often consist of a great number of trainable parameters for extracting powerful features from given datasets. On one hand, massive trainable parameters signifi…
Widening and Squeezing: Towards Accurate and Efficient QNNs
Chuanjian Liu, Kai Han, Yunhe Wang +3
Quantization neural networks (QNNs) are very attractive to the industry because their extremely cheap calculation and storage overhead, but their performance is still worse than th…
On Positive-Unlabeled Classification in GAN
Tianyu Guo, Chang Xu, Jiajun Huang +4
This paper defines a positive and unlabeled classification problem for standard GANs, which then leads to a novel technique to stabilize the training of the discriminator in GANs.…
Discernible Image Compression
Zhaohui Yang, Yunhe Wang, Chang Xu +4
Image compression, as one of the fundamental low-level image processing tasks, is very essential for computer vision. Tremendous computing and storage resources can be preserved wi…