Differentiable Fine-grained Quantization for Deep Neural Network Compression

Abstract

Neural networks have shown great performance in cognitive tasks. When deploying network models on mobile devices with limited resources, weight quantization has been widely adopted. Binary quantization obtains the highest compression but usually results in big accuracy drop. In practice, 8-bit or 16-bit quantization is often used aiming at maintaining the same accuracy as the original 32-bit precision. We observe different layers have different accuracy sensitivity of quantization. Thus judiciously selecting different precision for different layers/structures can potentially produce more efficient models compared to traditional quantization methods by striking a better balance between accuracy and compression rate. In this work, we propose a fine-grained quantization approach for deep neural network compression by relaxing the search space of quantization bitwidth from discrete to a continuous domain. The proposed approach applies gradient descend based optimization to generate a mixed-precision quantization scheme that outperforms the accuracy of traditional quantization methods under the same compression rate.

arXiv commentHsin-Pai Cheng, Yuanjun Huang and Xuyang Guo contributed equally and are co-first authors for this paper. This work has been accepted by NIPS 2018 Workshop on Compact Deep Neural Network Representation with Industrial Applications, Montreal, Canada

References in corpus (6)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• DARTS: Differentiable Architecture Search
• ProxQuant: Quantized Neural Networks via Proximal Operators
• Progressive Weight Pruning of Deep Neural Networks using ADMM

Cited by in corpus (6)

• Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?
• Progressive DNN Compression: A Key to Achieve Ultra-High Weight Pruning and Quantization Rates using ADMM
• Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms
• Temporal-Coded Deep Spiking Neural Network with Easy Training and Robust Performance
• Toward Extremely Low Bit and Lossless Accuracy in DNNs with Progressive ADMM
• Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bit-wise Regularization