Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision

Abstract

We consider the post-training quantization problem, which discretizes the weights of pre-trained deep neural networks without re-training the model. We propose multipoint quantization, a quantization method that approximates a full-precision weight vector using a linear combination of multiple vectors of low-bit numbers; this is in contrast to typical quantization methods that approximate each weight using a single low precision number. Computationally, we construct the multipoint quantization with an efficient greedy selection procedure, and adaptively decides the number of low precision points on each quantized weight vector based on the error of its output. This allows us to achieve higher precision levels for important weights that greatly influence the outputs, yielding an 'effect of mixed precision' but without physical mixed precision implementations (which requires specialized hardware accelerators). Empirically, our method can be implemented by common operands, bringing almost no memory and computation overhead. We show that our method outperforms a range of state-of-the-art methods on ImageNet classification and it can be generalized to more challenging tasks like PASCAL VOC object detection.

arXiv commentAccepted by AAAI2021

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Trained Ternary Quantization
• WRPN: Wide Reduced-Precision Networks
• Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
• Resiliency of Deep Neural Networks under Quantization
• Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks
• Relaxed Quantization for Discretized Neural Networks
• Recent Advances in Efficient Computation of Deep Convolutional Neural Networks

Cited by in corpus (3)

• Weight Equalizing Shift Scaler-Coupled Post-training Quantization
• Exploring Neural Networks Quantization via Layer-Wise Quantization Analysis
• Dual Precision Deep Neural Network