A High-Performance Adaptive Quantization Approach for Edge CNN Applications

Abstract

Recent convolutional neural network (CNN) development continues to advance the state-of-the-art model accuracy for various applications. However, the enhanced accuracy comes at the cost of substantial memory bandwidth and storage requirements and demanding computational resources. Although in the past the quantization methods have effectively reduced the deployment cost for edge devices, it suffers from significant information loss when processing the biased activations of contemporary CNNs. In this paper, we hence introduce an adaptive high-performance quantization method to resolve the issue of biased activation by dynamically adjusting the scaling and shifting factors based on the task loss. Our proposed method has been extensively evaluated on image classification models (ResNet-18/34/50, MobileNet-V2, EfficientNet-B0) with ImageNet dataset, object detection model (YOLO-V4) with COCO dataset, and language models with PTB dataset. The results show that our 4-bit integer (INT4) quantization models achieve better accuracy than the state-of-the-art 4-bit models, and in some cases, even surpass the golden full-precision models. The final designs have been successfully deployed onto extremely resource-constrained edge devices for many practical applications.

References in corpus (20)

• YOLOv4: Optimal Speed and Accuracy of Object Detection
• An overview of gradient descent optimization algorithms
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• Empirical Evaluation of Rectified Activations in Convolutional Network
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Searching for Activation Functions
• Quantizing deep convolutional networks for efficient inference: A whitepaper
• Ternary Weight Networks
• Trained Ternary Quantization
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Pruning Filters for Efficient ConvNets
• Fixed Point Quantization of Deep Convolutional Networks
• Learning Structured Sparsity in Deep Neural Networks
• A deep convolutional neural network model for rapid prediction of fluvial flood inundation
• Learned Step Size Quantization
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
• Scalable Methods for 8-bit Training of Neural Networks
• NICE: Noise Injection and Clamping Estimation for Neural Network Quantization