Weight Equalizing Shift Scaler-Coupled Post-training Quantization
arXiv:2008.05767
Abstract
Post-training, layer-wise quantization is preferable because it is free from retraining and is hardware-friendly. Nevertheless, accuracy degradation has occurred when a neural network model has a big difference of per-out-channel weight ranges. In particular, the MobileNet family has a tragedy drop in top-1 accuracy from 70.60% ~ 71.87% to 0.1% on the ImageNet dataset after 8-bit weight quantization. To mitigate this significant accuracy reduction, we propose a new weight equalizing shift scaler, i.e. rescaling the weight range per channel by a 4-bit binary shift, prior to a layer-wise quantization. To recover the original output range, inverse binary shifting is efficiently fused to the existing per-layer scale compounding in the fixed-computing convolutional operator of the custom neural processing unit. The binary shift is a key feature of our algorithm, which significantly improved the accuracy performance without impeding the memory footprint. As a result, our proposed method achieved a top-1 accuracy of 69.78% ~ 70.96% in MobileNets and showed robust performance in varying network models and tasks, which is competitive to channel-wise quantization results.
arXiv comment9 pages, 4 figures, 4 tables
References in corpus (8)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Binarized Neural Networks
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
- DeepTwist: Learning Model Compression via Occasional Weight Distortion
- Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision