Channel-wise Hessian Aware trace-Weighted Quantization of Neural Networks

Abstract

Second-order information has proven to be very effective in determining the redundancy of neural network weights and activations. Recent paper proposes to use Hessian traces of weights and activations for mixed-precision quantization and achieves state-of-the-art results. However, prior works only focus on selecting bits for each layer while the redundancy of different channels within a layer also differ a lot. This is mainly because the complexity of determining bits for each channel is too high for original methods. Here, we introduce Channel-wise Hessian Aware trace-Weighted Quantization (CW-HAWQ). CW-HAWQ uses Hessian trace to determine the relative sensitivity order of different channels of activations and weights. What's more, CW-HAWQ proposes to use deep Reinforcement learning (DRL) Deep Deterministic Policy Gradient (DDPG)-based agent to find the optimal ratios of different quantization bits and assign bits to channels according to the Hessian trace order. The number of states in CW-HAWQ is much smaller compared with traditional AutoML based mix-precision methods since we only need to search ratios for the quantization bits. Compare CW-HAWQ with state-of-the-art shows that we can achieve better results for multiple networks.

References in corpus (12)

• Deep Residual Learning for Image Recognition
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Trained Ternary Quantization
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Post-training 4-bit quantization of convolution networks for rapid-deployment
• HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
• Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)

Cited by in corpus (4)

• Network Quantization with Element-wise Gradient Scaling
• Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization
• RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions
• Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization