ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks

Abstract

Deep Neural Networks (DNNs) typically require massive amount of computation resource in inference tasks for computer vision applications. Quantization can significantly reduce DNN computation and storage by decreasing the bitwidth of network encodings. Recent research affirms that carefully selecting the quantization levels for each layer can preserve the accuracy while pushing the bitwidth below eight bits. However, without arduous manual effort, this deep quantization can lead to significant accuracy loss, leaving it in a position of questionable utility. As such, deep quantization opens a large hyper-parameter space (bitwidth of the layers), the exploration of which is a major challenge. We propose a systematic approach to tackle this problem, by automating the process of discovering the quantization levels through an end-to-end deep reinforcement learning framework (ReLeQ). We adapt policy optimization methods to the problem of quantization, and focus on finding the best design decisions in choosing the state and action spaces, network architecture and training framework, as well as the tuning of various hyperparamters. We show how ReLeQ can balance speed and quality, and provide an asymmetric general solution for quantization of a large variety of deep networks (AlexNet, CIFAR-10, LeNet, MobileNet-V1, ResNet-20, SVHN, and VGG-11) that virtually preserves the accuracy (=< 0.3% loss) while minimizing the computation and storage cost. With these DNNs, ReLeQ enables conventional hardware to achieve 2.2x speedup over 8-bit execution. Similarly, a custom DNN accelerator achieves 2.0x speedup and energy reduction compared to 8-bit runs. These encouraging results mark ReLeQ as the initial step towards automating the deep quantization of neural networks.

arXiv commentPresented as a spotlight paper at NeurIPS Workshop on ML for Systems 2018

References in corpus (23)

• Scikit-learn: Machine Learning in Python
• Natural Language Processing (almost) from Scratch
• Neural Architecture Search with Reinforcement Learning
• Neural Architecture Search: A Survey
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• Mixed Precision Training
• Binarized Neural Networks
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Ternary Weight Networks
• Trained Ternary Quantization
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Efficient Neural Architecture Search via Parameter Sharing
• NAS-Bench-101: Towards Reproducible Neural Architecture Search
• Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
• WRPN: Wide Reduced-Precision Networks
• Are All Layers Created Equal?
• HAQ: Hardware-Aware Automated Quantization with Mixed Precision
• A Unified Framework of DNN Weight Pruning and Weight Clustering/Quantization Using ADMM
• Heterogeneous Bitwidth Binarization in Convolutional Neural Networks
• Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks
• SinReQ: Generalized Sinusoidal Regularization for Low-Bitwidth Deep Quantized Training

Cited by in corpus (21)

• Mixed Precision DNNs: All you need is a good parametrization
• Towards Efficient Training for Neural Network Quantization
• AutoQ: Automated Kernel-Wise Neural Network Quantization
• Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation
• AdaBits: Neural Network Quantization with Adaptive Bit-Widths
• BitPruning: Learning Bitlengths for Aggressive and Accurate Quantization
• Reinforcement Learning and Adaptive Sampling for Optimized DNN Compilation
• Efficient Bitwidth Search for Practical Mixed Precision Neural Network
• GeneCAI: Genetic Evolution for Acquiring Compact AI
• Rethinking Differentiable Search for Mixed-Precision Neural Networks
• CNN2Gate: Toward Designing a General Framework for Implementation of Convolutional Neural Networks on FPGA
• Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization
• Ordering Chaos: Memory-Aware Scheduling of Irregularly Wired Neural Networks for Edge Devices
• Differentiable Joint Pruning and Quantization for Hardware Efficiency
• Automated Model Compression by Jointly Applied Pruning and Quantization
• SinReQ: Generalized Sinusoidal Regularization for Low-Bitwidth Deep Quantized Training
• Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks
• Automatic Neural Network Compression by Sparsity-Quantization Joint Learning: A Constrained Optimization-based Approach
• MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantization
• Exploring Neural Networks Quantization via Layer-Wise Quantization Analysis
• Modulating Regularization Frequency for Efficient Compression-Aware Model Training