All You Need is a Few Shifts: Designing Efficient Convolutional Neural Networks for Image Classification
arXiv:1903.05285
Abstract
Shift operation is an efficient alternative over depthwise separable convolution. However, it is still bottlenecked by its implementation manner, namely memory movement. To put this direction forward, a new and novel basic component named Sparse Shift Layer (SSL) is introduced in this paper to construct efficient convolutional neural networks. In this family of architectures, the basic block is only composed by 1x1 convolutional layers with only a few shift operations applied to the intermediate feature maps. To make this idea feasible, we introduce shift operation penalty during optimization and further propose a quantization-aware shift learning method to impose the learned displacement more friendly for inference. Extensive ablation studies indicate that only a few shift operations are sufficient to provide spatial information communication. Furthermore, to maximize the role of SSL, we redesign an improved network architecture to Fully Exploit the limited capacity of neural Network (FE-Net). Equipped with SSL, this network can achieve 75.0% top-1 accuracy on ImageNet with only 563M M-Adds. It surpasses other counterparts constructed by depthwise separable convolution and the networks searched by NAS in terms of accuracy and practical speed.
arXiv commentCVPR2019
References in corpus (8)
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
- Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
- DARTS: Differentiable Architecture Search
- Compressing Deep Convolutional Networks using Vector Quantization
- Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
- Deep Model Compression: Distilling Knowledge from Noisy Teachers