Post-Training Piecewise Linear Quantization for Deep Neural Networks
arXiv:2002.00104
Abstract
Quantization plays an important role in the energy-efficient deployment of deep neural networks on resource-limited devices. Post-training quantization is highly desirable since it does not require retraining or access to the full training dataset. The well-established uniform scheme for post-training quantization achieves satisfactory results by converting neural networks from full-precision to 8-bit fixed-point integers. However, it suffers from significant performance degradation when quantizing to lower bit-widths. In this paper, we propose a piecewise linear quantization (PWLQ) scheme to enable accurate approximation for tensor values that have bell-shaped distributions with long tails. Our approach breaks the entire quantization range into non-overlapping regions for each tensor, with each region being assigned an equal number of quantization levels. Optimal breakpoints that divide the entire range are found by minimizing the quantization error. Compared to state-of-the-art post-training quantization methods, experimental results show that our proposed method achieves superior performance on image classification, semantic segmentation, and object detection with minor overhead.
References in corpus (23)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Wide Residual Networks
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- Compressing Deep Convolutional Networks using Vector Quantization
- Mixed Precision Training
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- Trained Ternary Quantization
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Pruning Filters for Efficient ConvNets
- Fixed Point Quantization of Deep Convolutional Networks
- Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
- Convolutional Neural Networks using Logarithmic Data Representation
- Model compression via distillation and quantization
- Stochastic Activation Pruning for Robust Adversarial Defense
- Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
- Post-training 4-bit quantization of convolution networks for rapid-deployment
- Deep Learning with Low Precision by Half-wave Gaussian Quantization
- Quantization for Rapid Deployment of Deep Neural Networks
- Fighting Quantization Bias With Bias