One Model for All Quantization: A Quantized Network Supporting Hot-Swap Bit-Width Adjustment
arXiv:2105.01353
Abstract
As an effective technique to achieve the implementation of deep neural networks in edge devices, model quantization has been successfully applied in many practical applications. No matter the methods of quantization aware training (QAT) or post-training quantization (PTQ), they all depend on the target bit-widths. When the precision of quantization is adjusted, it is necessary to fine-tune the quantized model or minimize the quantization noise, which brings inconvenience in practical applications. In this work, we propose a method to train a model for all quantization that supports diverse bit-widths (e.g., form 8-bit to 1-bit) to satisfy the online quantization bit-width adjustment. It is hot-swappable that can provide specific quantization strategies for different candidates through multiscale quantization. We use wavelet decomposition and reconstruction to increase the diversity of weights, thus significantly improving the performance of each quantization candidate, especially at ultra-low bit-widths (e.g., 3-bit, 2-bit, and 1-bit). Experimental results on ImageNet and COCO show that our method can achieve accuracy comparable performance to dedicated models trained at the same precision.
References in corpus (12)
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
- Convolutional Neural Networks using Logarithmic Data Representation
- Defensive Quantization: When Efficiency Meets Robustness
- Robust Quantization: One Model to Rule Them All
- AdaBits: Neural Network Quantization with Adaptive Bit-Widths
- Rethinking Differentiable Search for Mixed-Precision Neural Networks
- Any-Precision Deep Neural Networks
- WaveSNet: Wavelet Integrated Deep Networks for Image Segmentation
- MWQ: Multiscale Wavelet Quantized Neural Networks
- Effective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization