Shift-based Primitives for Efficient Convolutional Neural Networks

Abstract

We propose a collection of three shift-based primitives for building efficient compact CNN-based networks. These three primitives (channel shift, address shift, shortcut shift) can reduce the inference time on GPU while maintains the prediction accuracy. These shift-based primitives only moves the pointer but avoids memory copy, thus very fast. For example, the channel shift operation is 12.7x faster compared to channel shuffle in ShuffleNet but achieves the same accuracy. The address shift and channel shift can be merged into the point-wise group convolution and invokes only a single kernel call, taking little time to perform spatial convolution and channel shift. Shortcut shift requires no time to realize residual connection through allocating space in advance. We blend these shift-based primitives with point-wise group convolution and built two inference-efficient CNN architectures named AddressNet and Enhanced AddressNet. Experiments on CIFAR100 and ImageNet datasets show that our models are faster and achieve comparable or better accuracy.

References in corpus (14)

• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Compressing Deep Convolutional Networks using Vector Quantization
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• Convolutional neural networks with low-rank regularization
• Channel Pruning for Accelerating Very Deep Neural Networks
• Flattened Convolutional Neural Networks for Feedforward Acceleration
• Bayesian Dark Knowledge
• Deep Model Compression: Distilling Knowledge from Noisy Teachers
• Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions
• Keynote: Small Neural Nets Are Beautiful: Enabling Embedded Systems with Small Deep-Neural-Network Architectures
• Single Image Super-resolution via a Lightweight Residual Convolutional Neural Network
• Sparsely Aggregated Convolutional Networks

Cited by in corpus (6)

• TSM: Temporal Shift Module for Efficient Video Understanding
• Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs
• ShiftAddNet: A Hardware-Inspired Deep Network
• AdderNet: Do We Really Need Multiplications in Deep Learning?
• Structured Pruning for Efficient ConvNets via Incremental Regularization
• Structured Convolutions for Efficient Neural Network Design