Quantization for Rapid Deployment of Deep Neural Networks

Abstract

This paper aims at rapid deployment of the state-of-the-art deep neural networks (DNNs) to energy efficient accelerators without time-consuming fine tuning or the availability of the full datasets. Converting DNNs in full precision to limited precision is essential in taking advantage of the accelerators with reduced memory footprint and computation power. However, such a task is not trivial since it often requires the full training and validation datasets for profiling the network statistics and fine tuning the networks to recover the accuracy lost after quantization. To address these issues, we propose a simple method recognizing channel-level distribution to reduce the quantization-induced accuracy loss and minimize the required image samples for profiling. We evaluated our method on eleven networks trained on the ImageNet classification benchmark and a network trained on the Pascal VOC object detection benchmark. The results prove that the networks can be quantized into 8-bit integer precision without fine tuning.

References in corpus (16)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• Deep Residual Learning for Image Recognition
• MobileNetV2: Inverted Residuals and Linear Bottlenecks
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Going Deeper with Convolutions
• Ternary Weight Networks
• Rethinking the Inception Architecture for Computer Vision
• YOLO9000: Better, Faster, Stronger
• Towards Accurate Binary Convolutional Neural Network
• Aggregated Residual Transformations for Deep Neural Networks
• Xception: Deep Learning with Depthwise Separable Convolutions
• Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks
• Towards the Limit of Network Quantization
• Towards Effective Low-bitwidth Convolutional Neural Networks

Cited by in corpus (14)

• Post-training 4-bit quantization of convolution networks for rapid-deployment
• Applications and Techniques for Fast Machine Learning in Science
• Robust Quantization: One Model to Rule Them All
• Feature Map Transform Coding for Energy-Efficient CNN Inference
• ZeroQ: A Novel Zero Shot Quantization Framework
• Post-Training Piecewise Linear Quantization for Deep Neural Networks
• VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference
• FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation
• CAT: Compression-Aware Training for bandwidth reduction
• Rethinking Floating Point Overheads for Mixed Precision DNN Accelerators
• Term Revealing: Furthering Quantization at Run Time on Quantized DNNs
• Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks
• HPTQ: Hardware-Friendly Post Training Quantization
• Neural Network Activation Quantization with Bitwise Information Bottlenecks