A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics
arXiv:2112.02834 · doi:10.1109/TMM.2021.3134158
Abstract
Quantizing the floating-point weights and activations of deep convolutional neural networks to fixed-point representation yields reduced memory footprints and inference time. Recently, efforts have been afoot towards zero-shot quantization that does not require original unlabelled training samples of a given task. These best-published works heavily rely on the learned batch normalization (BN) parameters to infer the range of the activations for quantization. In particular, these methods are built upon either empirical estimation framework or the data distillation approach, for computing the range of the activations. However, the performance of such schemes severely degrades when presented with a network that does not accommodate BN layers. In this line of thought, we propose a generalized zero-shot quantization (GZSQ) framework that neither requires original data nor relies on BN layer statistics. We have utilized the data distillation approach and leveraged only the pre-trained weights of the model to estimate enriched data for range calibration of the activations. To the best of our knowledge, this is the first work that utilizes the distribution of the pretrained weights to assist the process of zero-shot quantization. The proposed scheme has significantly outperformed the existing zero-shot works, e.g., an improvement of ~ 33% in classification accuracy for MobileNetV2 and several other models that are w & w/o BN layers, for a variety of tasks. We have also demonstrated the efficacy of the proposed work across multiple open-source quantization frameworks. Importantly, our work is the first attempt towards the post-training zero-shot quantization of futuristic unnormalized deep neural networks.
arXiv commentAccepted by IEEE Transactions on Multimedia
References in corpus (19)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- Distilling the Knowledge in a Neural Network
- Neural Architecture Search with Reinforcement Learning
- FitNets: Hints for Thin Deep Nets
- MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
- DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
- AMC: AutoML for Model Compression and Acceleration on Mobile Devices
- Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- PACT: Parameterized Clipping Activation for Quantized Neural Networks
- Bayesian Compression for Deep Learning
- Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
- Resiliency of Deep Neural Networks under Quantization
- A Survey of FPGA-Based Neural Network Accelerator
- Deep Isometric Learning for Visual Recognition
- Rate Distortion For Model Compression: From Theory To Practice
- CAT: Compression-Aware Training for bandwidth reduction