BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer

Abstract

As the applications of deep learning models on edge devices increase at an accelerating pace, fast adaptation to various scenarios with varying resource constraints has become a crucial aspect of model deployment. As a result, model optimization strategies with adaptive configuration are becoming increasingly popular. While single-shot quantized neural architecture search enjoys flexibility in both model architecture and quantization policy, the combined search space comes with many challenges, including instability when training the weight-sharing supernet and difficulty in navigating the exponentially growing search space. Existing methods tend to either limit the architecture search space to a small set of options or limit the quantization policy search space to fixed precision policies. To this end, we propose BatchQuant, a robust quantizer formulation that allows fast and stable training of a compact, single-shot, mixed-precision, weight-sharing supernet. We employ BatchQuant to train a compact supernet (offering over $10^{76}$ quantized subnets) within substantially fewer GPU hours than previous methods. Our approach, Quantized-for-all (QFA), is the first to seamlessly extend one-shot weight-sharing NAS supernet to support subnets with arbitrary ultra-low bitwidth mixed-precision quantization policies without retraining. QFA opens up new possibilities in joint hardware-aware neural architecture search and quantization. We demonstrate the effectiveness of our method on ImageNet and achieve SOTA Top-1 accuracy under a low complexity constraint ($<20$ MFLOPs). The code and models will be made publicly available at https://github.com/bhpfelix/QFA.

References in corpus (15)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Learned Step Size Quantization
• Single Path One-Shot Neural Architecture Search with Uniform Sampling
• Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
• Neural Architecture Transfer
• QKD: Quantization-aware Knowledge Distillation
• AtomNAS: Fine-Grained End-to-End Neural Architecture Search
• Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollers
• Joint Neural Architecture Search and Quantization
• Gradient ${\ell }_1$ Regularization for Quantization Robustness
• Any-Precision Deep Neural Networks

Cited by in corpus (1)

• Scaling Up Quantization-Aware Neural Architecture Search for Efficient Deep Learning on the Edge