Recent Advances in Convolutional Neural Network Acceleration

Abstract

In recent years, convolutional neural networks (CNNs) have shown great performance in various fields such as image classification, pattern recognition, and multi-media compression. Two of the feature properties, local connectivity and weight sharing, can reduce the number of parameters and increase processing speed during training and inference. However, as the dimension of data becomes higher and the CNN architecture becomes more complicated, the end-to-end approach or the combined manner of CNN is computationally intensive, which becomes limitation to CNN's further implementation. Therefore, it is necessary and urgent to implement CNN in a faster way. In this paper, we first summarize the acceleration methods that contribute to but not limited to CNN by reviewing a broad variety of research papers. We propose a taxonomy in terms of three levels, i.e.~structure level, algorithm level, and implementation level, for acceleration methods. We also analyze the acceleration methods in terms of CNN architecture compression, algorithm optimization, and hardware-based improvement. At last, we give a discussion on different perspectives of these acceleration and optimization methods within each level. The discussion shows that the methods in each level still have large exploration space. By incorporating such a wide range of disciplines, we expect to provide a comprehensive reference for researchers who are interested in CNN acceleration.

arXiv commentsubmitted to Neurocomputing

References in corpus (22)

• Deep Learning in Neural Networks: An Overview
• Distilling the Knowledge in a Neural Network
• ADADELTA: An Adaptive Learning Rate Method
• Training and Operation of an Integrated Neuromorphic Network Based on Metal-Oxide Memristors
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• BinaryConnect: Training Deep Neural Networks with binary weights during propagations
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
• Deep Learning with Limited Numerical Precision
• Ternary Weight Networks
• Predicting Parameters in Deep Learning
• Stochastic Pooling for Regularization of Deep Convolutional Neural Networks
• Speeding up Convolutional Neural Networks with Low Rank Expansions
• Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices
• Learning Structured Sparsity in Deep Neural Networks
• Convolutional neural networks with low-rank regularization
• Channel Pruning for Accelerating Very Deep Neural Networks
• Deep Image: Scaling up Image Recognition
• Fast Training of Convolutional Networks through FFTs
• Lossy Image Compression with Compressive Autoencoders
• Bitwise Neural Networks