A Learning Framework for n-bit Quantized Neural Networks toward FPGAs

Abstract

The quantized neural network (QNN) is an efficient approach for network compression and can be widely used in the implementation of FPGAs. This paper proposes a novel learning framework for n-bit QNNs, whose weights are constrained to the power of two. To solve the gradient vanishing problem, we propose a reconstructed gradient function for QNNs in back-propagation algorithm that can directly get the real gradient rather than estimating an approximate gradient of the expected loss. We also propose a novel QNN structure named n-BQ-NN, which uses shift operation to replace the multiply operation and is more suitable for the inference on FPGAs. Furthermore, we also design a shift vector processing element (SVPE) array to replace all 16-bit multiplications with SHIFT operations in convolution operation on FPGAs. We also carry out comparable experiments to evaluate our framework. The experimental results show that the quantized models of ResNet, DenseNet and AlexNet through our learning framework can achieve almost the same accuracies with the original full-precision models. Moreover, when using our learning framework to train our n-BQ-NN from scratch, it can achieve state-of-the-art results compared with typical low-precision QNNs. Experiments on Xilinx ZCU102 platform show that our n-BQ-NN with our SVPE can execute 2.9 times faster than with the vector processing element (VPE) in inference. As the SHIFT operation in our SVPE array will not consume Digital Signal Processings (DSPs) resources on FPGAs, the experiments have shown that the use of SVPE array also reduces average energy consumption to 68.7% of the VPE array with 16-bit.

This paper has been accepted for publication in the IEEE Transactions on Neural Networks and Learning Systems

References in corpus (23)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Fully Convolutional Networks for Semantic Segmentation
• Striving for Simplicity: The All Convolutional Net
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• FitNets: Hints for Thin Deep Nets
• Wide Residual Networks
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Training Very Deep Networks
• Deeply-Supervised Nets
• Ternary Weight Networks
• Trained Ternary Quantization
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Pruning Filters for Efficient ConvNets
• FractalNet: Ultra-Deep Neural Networks without Residuals
• Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights
• Convolutional Neural Networks Applied to House Numbers Digit Classification
• Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)
• Precision Highway for Ultra Low-Precision Quantization

Cited by in corpus (4)

• Towards Efficient In-memory Computing Hardware for Quantized Neural Networks: State-of-the-art, Open Challenges and Perspectives
• Computer Vision Model Compression Techniques for Embedded Systems: A Survey
• Ternary Compression for Communication-Efficient Federated Learning
• Adaptive Resolution Inference (ARI): Energy-Efficient Machine Learning for Internet of Things