DSConv: Efficient Convolution Operator

Abstract

Quantization is a popular way of increasing the speed and lowering the memory usage of Convolution Neural Networks (CNNs). When labelled training data is available, network weights and activations have successfully been quantized down to 1-bit. The same cannot be said about the scenario when labelled training data is not available, e.g. when quantizing a pre-trained model, where current approaches show, at best, no loss of accuracy at 8-bit quantizations. We introduce DSConv, a flexible quantized convolution operator that replaces single-precision operations with their far less expensive integer counterparts, while maintaining the probability distributions over both the kernel weights and the outputs. We test our model as a plug-and-play replacement for standard convolution on most popular neural network architectures, ResNet, DenseNet, GoogLeNet, AlexNet and VGG-Net and demonstrate state-of-the-art results, with less than 1% loss of accuracy, without retraining, using only 4-bit quantization. We also show how a distillation-based adaptation stage with unlabelled data can improve results even further.

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Do Deep Nets Really Need to be Deep?
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Ternary Weight Networks
• Trained Ternary Quantization
• Compressing Neural Networks with the Hashing Trick
• Training and Inference with Integers in Deep Neural Networks
• Training deep neural networks with low precision multiplications
• Model compression via distillation and quantization
• WRPN: Wide Reduced-Precision Networks
• Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
• Post-training 4-bit quantization of convolution networks for rapid-deployment
• Training DNNs with Hybrid Block Floating Point