Fighting Quantization Bias With Bias

Abstract

Low-precision representation of deep neural networks (DNNs) is critical for efficient deployment of deep learning application on embedded platforms, however, converting the network to low precision degrades its performance. Crucially, networks that are designed for embedded applications usually suffer from increased degradation since they have less redundancy. This is most evident for the ubiquitous MobileNet architecture which requires a costly quantization-aware training cycle to achieve acceptable performance when quantized to 8-bits. In this paper, we trace the source of the degradation in MobileNets to a shift in the mean activation value. This shift is caused by an inherent bias in the quantization process which builds up across layers, shifting all network statistics away from the learned distribution. We show that this phenomenon happens in other architectures as well. We propose a simple remedy - compensating for the quantization induced shift by adding a constant to the additive bias term of each channel. We develop two simple methods for estimating the correction constants - one using iterative evaluation of the quantized network and one where the constants are set using a short training phase. Both methods are fast and require only a small amount of unlabeled data, making them appealing for rapid deployment of neural networks. Using the above methods we are able to match the performance of training-based quantization of MobileNets at a fraction of the cost.

Accepted to ECV workshop at CVPR2019

References in corpus (9)

• Distilling the Knowledge in a Neural Network
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
• Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference
• Quantizing Convolutional Neural Networks for Low-Power High-Throughput Inference Engines

Cited by in corpus (11)

• Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programming
• Up or Down? Adaptive Rounding for Post-Training Quantization
• Robust Quantization: One Model to Rule Them All
• Post-Training Piecewise Linear Quantization for Deep Neural Networks
• Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks
• A White Paper on Neural Network Quantization
• Generative Zero-shot Network Quantization
• Non-Blocking Simultaneous Multithreading: Embracing the Resiliency of Deep Neural Networks
• Exploring Neural Networks Quantization via Layer-Wise Quantization Analysis
• Q-Rater: Non-Convex Optimization for Post-Training Uniform Quantization
• HPTQ: Hardware-Friendly Post Training Quantization