7 papers
Incorporating Convolution Designs into Visual Transformers
Kun Yuan, Shaopeng Guo, Ziwei Liu +3
Motivated by the success of Transformers in natural language processing (NLP) tasks, there emerge some attempts (e.g., ViT and DeiT) to apply Transformers to the vision domain. How…
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
Aojun Zhou, Yukun Ma, Junnan Zhu +5
Sparsity in Deep Neural Networks (DNNs) has been widely studied to compress and accelerate the models on resource-constrained environments. It can be generally categorized into uns…
Dynamic Graph: Learning Instance-aware Connectivity for Neural Networks
Kun Yuan, Quanquan Li, Dapeng Chen +2
One practice of employing deep neural networks is to apply the same architecture to all the input instances. However, a fixed architecture may not be representative enough for data…
Scale Calibrated Training: Improving Generalization of Deep Networks via Scale-Specific Normalization
Zhuoran Yu, Aojun Zhou, Yukun Ma +3
Standard convolutional neural networks(CNNs) require consistent image resolutions in both training and testing phase. However, in practice, testing with smaller image sizes is nece…
HBONet: Harmonious Bottleneck on Two Orthogonal Dimensions
Duo Li, Aojun Zhou, Anbang Yao
MobileNets, a class of top-performing convolutional neural network architectures in terms of accuracy and efficiency trade-off, are increasingly used in many resourceaware vision a…
Deeply-supervised Knowledge Synergy
Dawei Sun, Anbang Yao, Aojun Zhou +1
Convolutional Neural Networks (CNNs) have become deeper and more complicated compared with the pioneering AlexNet. However, current prevailing training scheme follows the previous…