Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization
arXiv:2103.02904
Abstract
Since model quantization helps to reduce the model size and computation latency, it has been successfully applied in many applications of mobile phones, embedded devices and smart chips. The mixed-precision quantization model can match different quantization bit-precisions according to the sensitivity of different layers to achieve great performance. However, it is a difficult problem to quickly determine the quantization bit-precision of each layer in deep neural networks according to some constraints (e.g., hardware resources, energy consumption, model size and computation latency). To address this issue, we propose a novel sequential single path search (SSPS) method for mixed-precision quantization,in which the given constraints are introduced into its loss function to guide searching process. A single path search cell is used to combine a fully differentiable supernet, which can be optimized by gradient-based algorithms. Moreover, we sequentially determine the candidate precisions according to the selection certainties to exponentially reduce the search space and speed up the convergence of searching process. Experiments show that our method can efficiently search the mixed-precision models for different architectures (e.g., ResNet-20, 18, 34, 50 and MobileNet-V2) and datasets (e.g., CIFAR-10, ImageNet and COCO) under given constraints, and our experimental results verify that SSPS significantly outperforms their uniform counterparts.
References in corpus (27)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Categorical Reparameterization with Gumbel-Softmax
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Binarized Neural Networks
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Efficient Neural Architecture Search via Parameter Sharing
- Convolutional Neural Networks using Logarithmic Data Representation
- Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
- SNAS: Stochastic Neural Architecture Search
- Hierarchical Representations for Efficient Architecture Search
- Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search
- MaskConnect: Connectivity Learning by Gradient Descent
- Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks
- HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
- ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks
- AutoQ: Automated Kernel-Wise Neural Network Quantization
- ZeroQ: A Novel Zero Shot Quantization Framework
- AdaBits: Neural Network Quantization with Adaptive Bit-Widths
- Switchable Precision Neural Networks
- Densely Connected Search Space for More Flexible Neural Architecture Search
- Rethinking Differentiable Search for Mixed-Precision Neural Networks
- Channel-wise Hessian Aware trace-Weighted Quantization of Neural Networks
- Any-Precision Deep Neural Networks
- Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks
- Search What You Want: Barrier Panelty NAS for Mixed Precision Quantization