Differentiable Dynamic Quantization with Mixed Precision and Adaptive Resolution
arXiv:2106.02295
Abstract
Model quantization is challenging due to many tedious hyper-parameters such as precision (bitwidth), dynamic range (minimum and maximum discrete values) and stepsize (interval between discrete values). Unlike prior arts that carefully tune these values, we present a fully differentiable approach to learn all of them, named Differentiable Dynamic Quantization (DDQ), which has several benefits. (1) DDQ is able to quantize challenging lightweight architectures like MobileNets, where different layers prefer different quantization parameters. (2) DDQ is hardware-friendly and can be easily implemented using low-precision matrix-vector multiplication, making it capable in many hardware such as ARM. (3) Extensive experiments show that DDQ outperforms prior arts on many networks and benchmarks, especially when models are already efficient and compact. e.g., DDQ is the first approach that achieves lossless 4-bit quantization for MobileNetV2 on ImageNet.
arXiv commentAccepted by ICML 2021
References in corpus (15)
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- Trained Ternary Quantization
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Convolutional Neural Networks using Logarithmic Data Representation
- WRPN: Wide Reduced-Precision Networks
- Mixed Precision DNNs: All you need is a good parametrization
- Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference
- Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)
- NICE: Noise Injection and Clamping Estimation for Neural Network Quantization
- Towards Efficient Training for Neural Network Quantization
- Efficient non-uniform quantizer for quantized neural network targeting reconfigurable hardware