FrostNet: Towards Quantization-Aware Network Architecture Search
arXiv:2006.09679
Abstract
INT8 quantization has become one of the standard techniques for deploying convolutional neural networks (CNNs) on edge devices to reduce the memory and computational resource usages. By analyzing quantized performances of existing mobile-target network architectures, we can raise an issue regarding the importance of network architecture for optimal INT8 quantization. In this paper, we present a new network architecture search (NAS) procedure to find a network that guarantees both full-precision (FLOAT32) and quantized (INT8) performances. We first propose critical but straightforward optimization method which enables quantization-aware training (QAT) : floating-point statistic assisting (StatAssist) and stochastic gradient boosting (GradBoost). By integrating the gradient-based NAS with StatAssist and GradBoost, we discovered a quantization-efficient network building block, Frost bottleneck. Furthermore, we used Frost bottleneck as the building block for hardware-aware NAS to obtain quantization-efficient networks, FrostNets, which show improved quantization performances compared to other mobile-target networks while maintaining competitive FLOAT32 performance. Our FrostNets achieve higher recognition accuracy than existing CNNs with comparable latency when quantized, due to higher latency reduction rate (average 65%).
References in corpus (14)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Distilling the Knowledge in a Neural Network
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
- Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- Training with Quantization Noise for Extreme Model Compression
- NSML: Meet the MLaaS platform with a real-world case study
- Tiny-DSOD: Lightweight Object Detection for Resource-Restricted Usages
- And the Bit Goes Down: Revisiting the Quantization of Neural Networks