Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks

Abstract

Hardware-friendly network quantization (e.g., binary/uniform quantization) can efficiently accelerate the inference and meanwhile reduce memory consumption of the deep neural networks, which is crucial for model deployment on resource-limited devices like mobile phones. However, due to the discreteness of low-bit quantization, existing quantization methods often face the unstable training process and severe performance degradation. To address this problem, in this paper we propose Differentiable Soft Quantization (DSQ) to bridge the gap between the full-precision and low-bit networks. DSQ can automatically evolve during training to gradually approximate the standard quantization. Owing to its differentiable property, DSQ can help pursue the accurate gradients in backward propagation, and reduce the quantization loss in forward process with an appropriate clipping range. Extensive experiments over several popular network structures show that training low-bit neural networks with DSQ can consistently outperform state-of-the-art quantization methods. Besides, our first efficient implementation for deploying 2 to 4-bit DSQ on devices with ARM architecture achieves up to 1.7$\times$ speed up, compared with the open-source 8-bit high-performance inference framework NCNN. [31]

arXiv commentIEEE ICCV 2019

References in corpus (14)

• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Binarized Neural Networks
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Ternary Weight Networks
• Trained Ternary Quantization
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Convolutional Neural Networks using Logarithmic Data Representation
• Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
• Post-training 4-bit quantization of convolution networks for rapid-deployment
• Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference
• Deep Neural Network Compression with Single and Multiple Level Quantization

Cited by in corpus (11)

• Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks
• Training Binary Neural Networks with Real-to-Binary Convolutions
• BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction
• Searching for Low-Bit Weights in Quantized Neural Networks
• BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activations
• Towards Unified INT8 Training for Convolutional Neural Network
• HCM: Hardware-Aware Complexity Metric for Neural Network Architectures
• Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks
• MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantization
• BiDet: An Efficient Binarized Object Detector
• MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing