EasyQuant: Post-training Quantization via Scale Optimization

Abstract

The 8 bits quantization has been widely applied to accelerate network inference in various deep learning applications. There are two kinds of quantization methods, training-based quantization and post-training quantization. Training-based approach suffers from a cumbersome training process, while post-training quantization may lead to unacceptable accuracy drop. In this paper, we present an efficient and simple post-training method via scale optimization, named EasyQuant (EQ),that could obtain comparable accuracy with the training-based method.Specifically, we first alternately optimize scales of weights and activations for all layers target at convolutional outputs to further obtain the high quantization precision. Then, we lower down bit width to INT7 both for weights and activations, and adopt INT16 intermediate storage and integer Winograd convolution implementation to accelerate inference.Experimental results on various computer vision tasks show that EQ outperforms the TensorRT method and can achieve near INT8 accuracy in 7 bits width post-training.

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments
• WRPN: Wide Reduced-Precision Networks
• Post-training 4-bit quantization of convolution networks for rapid-deployment
• Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks
• Bit Efficient Quantization for Deep Neural Networks
• Neural Networks Weights Quantization: Target None-retraining Ternary (TNT)

Cited by in corpus (6)

• Towards Accurate Post-Training Quantization for Vision Transformer
• LLM-FP4: 4-Bit Floating-Point Quantized Transformers
• Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies
• Selective Focus: Investigating Semantics Sensitivity in Post-training Quantization for Lane Detection
• PTQ-SL: Exploring the Sub-layerwise Post-training Quantization
• Faster Convolution Inference Through Using Pre-Calculated Lookup Tables