LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks

Abstract

Although weight and activation quantization is an effective approach for Deep Neural Network (DNN) compression and has a lot of potentials to increase inference speed leveraging bit-operations, there is still a noticeable gap in terms of prediction accuracy between the quantized model and the full-precision model. To address this gap, we propose to jointly train a quantized, bit-operation-compatible DNN and its associated quantizers, as opposed to using fixed, handcrafted quantization schemes such as uniform or logarithmic quantization. Our method for learning the quantizers applies to both network weights and activations with arbitrary-bit precision, and our quantizers are easy to train. The comprehensive experiments on CIFAR-10 and ImageNet datasets show that our method works consistently well for various network structures such as AlexNet, VGG-Net, GoogLeNet, ResNet, and DenseNet, surpassing previous quantization methods in terms of accuracy by an appreciable margin. Code available at https://github.com/Microsoft/LQ-Nets

arXiv commentECCV'18 (European Conference on Computer Vision); Main paper + suppl. material

References in corpus (1)

• Some Improvements on Deep Convolutional Neural Network Based Image Classification

Cited by in corpus (22)

• Bringing AI To Edge: From Deep Learning's Perspective
• The NLP Cookbook: Modern Recipes for Transformer based Deep Learning Architectures
• Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks
• NICE: Noise Injection and Clamping Estimation for Neural Network Quantization
• Searching for Low-Bit Weights in Quantized Neural Networks
• AdaptivFloat: A Floating-point based Data Type for Resilient Deep Learning Inference
• Towards Unified INT8 Training for Convolutional Neural Network
• Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentation
• Sparse Weight Activation Training
• Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network using Truncated Gaussian Approximation
• Learning Versatile Convolution Filters for Efficient Visual Recognition
• Bit Error Robustness for Energy-Efficient DNN Accelerators
• Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks
• Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
• BiQGEMM: Matrix Multiplication with Lookup Table For Binary-Coding-based Quantized DNNs
• DKM: Differentiable K-Means Clustering Layer for Neural Network Compression
• Knowledge distillation for optimization of quantized deep neural networks
• PTQ-SL: Exploring the Sub-layerwise Post-training Quantization
• From Quantized DNNs to Quantizable DNNs
• Binarized Weight Error Networks With a Transition Regularization Term
• Robustness-aware 2-bit quantization with real-time performance for neural network
• Quantized Proximal Averaging Network for Analysis Sparse Coding