A Survey on Methods and Theories of Quantized Neural Networks

Abstract

Deep neural networks are the state-of-the-art methods for many real-world tasks, such as computer vision, natural language processing and speech recognition. For all its popularity, deep neural networks are also criticized for consuming a lot of memory and draining battery life of devices during training and inference. This makes it hard to deploy these models on mobile or embedded devices which have tight resource constraints. Quantization is recognized as one of the most effective approaches to satisfy the extreme memory requirements that deep neural network models demand. Instead of adopting 32-bit floating point format to represent weights, quantized representations store weights using more compact formats such as integers or even binary numbers. Despite a possible degradation in predictive performance, quantization provides a potential solution to greatly reduce the model size and the energy consumption. In this survey, we give a thorough review of different aspects of quantized neural networks. Current challenges and trends of quantized neural networks are also discussed.

17 pages, 8 figures

References in corpus (25)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Recent Trends in Deep Learning Based Natural Language Processing
• Compressing Deep Convolutional Networks using Vector Quantization
• Trained Ternary Quantization
• TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Towards Accurate Binary Convolutional Neural Network
• Training and Inference with Integers in Deep Neural Networks
• Model compression via distillation and quantization
• WRPN: Wide Reduced-Precision Networks
• Loss-aware Weight Quantization of Deep Networks
• Deep Learning with Low Precision by Half-wave Gaussian Quantization
• Effective Quantization Methods for Recurrent Neural Networks
• Ternary Neural Networks with Fine-Grained Quantization
• Towards the Limit of Network Quantization
• ShiftCNN: Generalized Low-Precision Architecture for Inference of Convolutional Neural Networks
• Recurrent Neural Networks With Limited Numerical Precision
• Rounding Methods for Neural Networks with Low Resolution Synaptic Weights
• Overcoming Challenges in Fixed Point Training of Deep Convolutional Networks
• Model compression as constrained optimization, with application to neural nets. Part II: quantization
• Adaptive Quantization for Deep Neural Network
• Network Sketching: Exploiting Binary Structure in Deep CNNs
• Sigma Delta Quantized Networks
• Training Bit Fully Convolutional Network for Fast Semantic Segmentation
• GXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework

Cited by in corpus (44)

• A Review on Deep Learning in UAV Remote Sensing
• Avoiding Overfitting: A Survey on Regularization Methods for Convolutional Neural Networks
• Structured Pruning for Deep Convolutional Neural Networks: A survey
• Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead
• Communication-Efficient Distributed Deep Learning: A Comprehensive Survey
• A Systematic Review on Model Watermarking for Neural Networks
• And the Bit Goes Down: Revisiting the Quantization of Neural Networks
• Up or Down? Adaptive Rounding for Post-Training Quantization
• Pruning and Quantization for Deep Neural Network Acceleration: A Survey
• Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
• On-Device Machine Learning: An Algorithms and Learning Theory Perspective
• Fusion-Catalyzed Pruning for Optimizing Deep Learning on Intelligent Edge Devices
• Depthwise Convolution is All You Need for Learning Multiple Visual Domains
• On the Adversarial Robustness of Quantized Neural Networks
• Fixed-point Quantization of Convolutional Neural Networks for Quantized Inference on Embedded Platforms
• Prominent characteristics of recurrent neuronal networks are robust against low synaptic weight resolution
• Bit Efficient Quantization for Deep Neural Networks
• Optimal training of integer-valued neural networks with mixed integer programming
• Mirror Descent View for Neural Network Quantization
• SYMOG: learning symmetric mixture of Gaussian modes for improved fixed-point quantization
• Bit Error Robustness for Energy-Efficient DNN Accelerators
• AdaFilter: Adaptive Filter Fine-tuning for Deep Transfer Learning
• QNNVerifier: A Tool for Verifying Neural Networks using SMT-Based Model Checking
• Scalar Quantization as Sparse Least Square Optimization
• Transformer Network for Semantically-Aware and Speech-Driven Upper-Face Generation
• Table-Based Neural Units: Fully Quantizing Networks for Multiply-Free Inference
• Shifted and Squeezed 8-bit Floating Point format for Low-Precision Training of Deep Neural Networks
• AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks
• Binarized Knowledge Graph Embeddings
• Binarized Canonical Polyadic Decomposition for Knowledge Graph Completion
• Unsupervised detection of semantic correlations in big data
• LANCE: Efficient Low-Precision Quantized Winograd Convolution for Neural Networks Based on Graphics Processing Units
• Information-Theoretic Understanding of Population Risk Improvement with Model Compression
• Verifying Quantized Neural Networks using SMT-Based Model Checking
• Demystifying and Generalizing BinaryConnect
• A Survey on Green Deep Learning
• Phoeni6: a Systematic Approach for Evaluating the Energy Consumption of Neural Networks
• Vector-Vector-Matrix Architecture: A Novel Hardware-Aware Framework for Low-Latency Inference in NLP Applications
• A Greedy Algorithm for Quantizing Neural Networks
• Differentiable Architecture Pruning for Transfer Learning
• Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization
• Smoothed Differential Privacy
• Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latency
• Volumization as a Natural Generalization of Weight Decay