Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search

Abstract

Recent work in network quantization has substantially reduced the time and space complexity of neural network inference, enabling their deployment on embedded and mobile devices with limited computational and memory resources. However, existing quantization methods often represent all weights and activations with the same precision (bit-width). In this paper, we explore a new dimension of the design space: quantizing different layers with different bit-widths. We formulate this problem as a neural architecture search problem and propose a novel differentiable neural architecture search (DNAS) framework to efficiently explore its exponential search space with gradient-based optimization. Experiments show we surpass the state-of-the-art compression of ResNet on CIFAR-10 and ImageNet. Our quantized models with 21.1x smaller model size or 103.9x lower computational cost can still outperform baseline quantized or even full precision models.

References in corpus (13)

• Neural Architecture Search with Reinforcement Learning
• Categorical Reparameterization with Gumbel-Softmax
• Improved Regularization of Convolutional Neural Networks with Cutout
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• DARTS: Differentiable Architecture Search
• AMC: AutoML for Model Compression and Acceleration on Mobile Devices
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Trained Ternary Quantization
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Convolutional Neural Fabrics
• Label Refinery: Improving ImageNet Classification through Label Progression
• Training Compact Neural Networks with Binary Weights and Low Precision Activations

Cited by in corpus (71)

• AutoML: A Survey of the State-of-the-Art
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• Exploring the Connection Between Binary and Spiking Neural Networks
• FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
• Bayesian Bits: Unifying Quantization and Pruning
• Applications and Techniques for Fast Machine Learning in Science
• A Comprehensive Survey on Hardware-Aware Neural Architecture Search
• HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
• Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
• Constructing Energy-efficient Mixed-precision Neural Networks through Principal Component Analysis for Edge Intelligence
• BinaryBERT: Pushing the Limit of BERT Quantization
• ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks
• HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
• Towards Efficient Training for Neural Network Quantization
• SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation
• Dustin: A 16-Cores Parallel Ultra-Low-Power Cluster with 2b-to-32b Fully Flexible Bit-Precision and Vector Lockstep Execution Mode
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• PSViT: Better Vision Transformer via Token Pooling and Attention Sharing
• ZeroQ: A Novel Zero Shot Quantization Framework
• HAWQV3: Dyadic Neural Network Quantization
• AdaBits: Neural Network Quantization with Adaptive Bit-Widths
• Learned Low Precision Graph Neural Networks
• N3H-Core: Neuron-designed Neural Network Accelerator via FPGA-based Heterogeneous Computing Cores
• Precision Gating: Improving Neural Network Efficiency with Dynamic Dual-Precision Activations
• Automatic Mixed-Precision Quantization Search of BERT
• I-BERT: Integer-only BERT Quantization
• BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer
• Elastic-InfoGAN: Unsupervised Disentangled Representation Learning in Class-Imbalanced Data
• VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference
• Efficient Bitwidth Search for Practical Mixed Precision Neural Network
• FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation
• Rethinking Differentiable Search for Mixed-Precision Neural Networks
• Automated Design Space Exploration for optimised Deployment of DNN on Arm Cortex-A CPUs
• Efficient Deep Neural Networks
• Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?
• Learning Versatile Convolution Filters for Efficient Visual Recognition
• Sharpness-aware Quantization for Deep Neural Networks
• Domain-Aware Dynamic Networks
• Rethinking Floating Point Overheads for Mixed Precision DNN Accelerators
• Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization
• MWQ: Multiscale Wavelet Quantized Neural Networks
• Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
• Learning Architectures for Binary Networks
• Adaptive Precision Training: Quantify Back Propagation in Neural Networks with Fixed-point Numbers
• Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks
• MetaMix: Meta-state Precision Searcher for Mixed-precision Activation Quantization
• Dynamic Network Quantization for Efficient Video Inference
• Confounding Tradeoffs for Neural Network Quantization
• DyCE: Dynamically Configurable Exiting for Deep Learning Compression and Real-time Scaling
• Training Quantized Neural Networks with a Full-precision Auxiliary Module
• Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators
• GDRQ: Group-based Distribution Reshaping for Quantization
• BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements
• Differentiable NAS Framework and Application to Ads CTR Prediction
• Data-free mixed-precision quantization using novel sensitivity metric
• GradFreeBits: Gradient Free Bit Allocation for Dynamic Low Precision Neural Networks
• Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths
• One Weight Bitwidth to Rule Them All
• InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks
• Real-Time Semantic Segmentation via Auto Depth, Downsampling Joint Decision and Feature Aggregation
• SoFAr: Shortcut-based Fractal Architectures for Binary Convolutional Neural Networks
• Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization
• RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions
• Adaptive Precision Training for Resource Constrained Devices
• Understanding and Overcoming the Challenges of Efficient Transformer Quantization
• Semi-Relaxed Quantization with DropBits: Training Low-Bit Neural Networks via Bit-wise Regularization
• PrivQuant: Communication-Efficient Private Inference with Quantized Network/Protocol Co-Optimization
• AutoQNN: An End-to-End Framework for Automatically Quantizing Neural Networks
• Auto-Split: A General Framework of Collaborative Edge-Cloud AI
• Accelerating Neural Network Inference by Overflow Aware Quantization