Resiliency of Deep Neural Networks under Quantization

Abstract

The complexity of deep neural network algorithms for hardware implementation can be much lowered by optimizing the word-length of weights and signals. Direct quantization of floating-point weights, however, does not show good performance when the number of bits assigned is small. Retraining of quantized networks has been developed to relieve this problem. In this work, the effects of retraining are analyzed for a feedforward deep neural network (FFDNN) and a convolutional neural network (CNN). The network complexity is controlled to know their effects on the resiliency of quantized networks by retraining. The complexity of the FFDNN is controlled by varying the unit size in each hidden layer and the number of layers, while that of the CNN is done by modifying the feature map configuration. We find that the performance gap between the floating-point and the retrain-based ternary (+1, 0, -1) weight neural networks exists with a fair amount in 'complexity limited' networks, but the discrepancy almost vanishes in fully complex networks whose capability is limited by the training data, rather than by the number of connections. This research shows that highly complex DNNs have the capability of absorbing the effects of severe weight quantization through retraining, but connection limited networks are less resilient. This paper also presents the effective compression ratio to guide the trade-off between the network size and the precision when the hardware resource is limited.

References in corpus (2)

• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• Fixed-Point Performance Analysis of Recurrent Neural Networks

Cited by in corpus (45)

• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review
• Convolutional Neural Networks using Logarithmic Data Representation
• Learned Step Size Quantization
• A Survey on Methods and Theories of Quantized Neural Networks
• WRPN: Wide Reduced-Precision Networks
• Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
• Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
• Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks
• Integrated Photonic Tensor Processing Unit for a Matrix Multiply: a Review
• SNIFF: Reverse Engineering of Neural Networks with Fault Attacks
• QKD: Quantization-aware Knowledge Distillation
• Structured Pruning of Deep Convolutional Neural Networks
• FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks
• LSQ+: Improving low-bit quantization through learnable offsets and better initialization
• Post-Training 4-bit Quantization on Embedding Tables
• Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?
• Generative Low-bitwidth Data Free Quantization
• FedDCT: Federated Learning of Large Convolutional Neural Networks on Resource Constrained Devices using Divide and Collaborative Training
• FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation
• ModelHub: Towards Unified Data and Lifecycle Management for Deep Learning
• Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit Integers
• A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics
• Bit Error Robustness for Energy-Efficient DNN Accelerators
• Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization
• Scaling Binarized Neural Networks on Reconfigurable Logic
• Diverse Sample Generation: Pushing the Limit of Generative Data-free Quantization
• Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precision
• On the efficient representation and execution of deep acoustic models
• Generative Zero-shot Network Quantization
• A GPU-Outperforming FPGA Accelerator Architecture for Binary Convolutional Neural Networks
• BitNet: Bit-Regularized Deep Neural Networks
• QuTiBench: Benchmarking Neural Networks on Heterogeneous Hardware
• Knowledge distillation for optimization of quantized deep neural networks
• OverQ: Opportunistic Outlier Quantization for Neural Network Accelerators
• Post-Training Quantization for Vision Transformer
• Fixed-point optimization of deep neural networks with adaptive step size retraining
• An Inter-Layer Weight Prediction and Quantization for Deep Neural Networks based on a Smoothly Varying Weight Hypothesis
• Dual Precision Deep Neural Network
• Q-Rater: Non-Convex Optimization for Post-Training Uniform Quantization
• Accuracy to Throughput Trade-offs for Reduced Precision Neural Networks on Reconfigurable Logic
• Optimize Deep Convolutional Neural Network with Ternarized Weights and High Accuracy
• NN2CAM: Automated Neural Network Mapping for Multi-Precision Edge Processing on FPGA-Based Cameras
• Robust error bounds for quantised and pruned neural networks