Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization

Abstract

Quantization is a widely used technique to compress and accelerate deep neural networks. However, conventional quantization methods use the same bit-width for all (or most of) the layers, which often suffer significant accuracy degradation in the ultra-low precision regime and ignore the fact that emergent hardware accelerators begin to support mixed-precision computation. Consequently, we present a novel and principled framework to solve the mixed-precision quantization problem in this paper. Briefly speaking, we first formulate the mixed-precision quantization as a discrete constrained optimization problem. Then, to make the optimization tractable, we approximate the objective function with second-order Taylor expansion and propose an efficient approach to compute its Hessian matrix. Finally, based on the above simplification, we show that the original problem can be reformulated as a Multiple-Choice Knapsack Problem (MCKP) and propose a greedy search algorithm to solve it efficiently. Compared with existing mixed-precision quantization works, our method is derived in a principled way and much more computationally efficient. Moreover, extensive experiments conducted on the ImageNet dataset and various kinds of network architectures also demonstrate its superiority over existing uniform and mixed-precision quantization approaches.

arXiv commentaccepted by ICCV 2021

References in corpus (16)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Distilling the Knowledge in a Neural Network
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Rethinking the Value of Network Pruning
• A Survey of Model Compression and Acceleration for Deep Neural Networks
• Ternary Weight Networks
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Towards Accurate Binary Convolutional Neural Network
• A Survey on Methods and Theories of Quantized Neural Networks
• Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
• HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
• Mixed Precision DNNs: All you need is a good parametrization
• AutoQ: Automated Kernel-Wise Neural Network Quantization
• Channel-wise Hessian Aware trace-Weighted Quantization of Neural Networks