FPGA-based Accelerators of Deep Learning Networks for Learning and Classification: A Review

Abstract

Due to recent advances in digital technologies, and availability of credible data, an area of artificial intelligence, deep learning, has emerged, and has demonstrated its ability and effectiveness in solving complex learning problems not possible before. In particular, convolution neural networks (CNNs) have demonstrated their effectiveness in image detection and recognition applications. However, they require intensive CPU operations and memory bandwidth that make general CPUs fail to achieve desired performance levels. Consequently, hardware accelerators that use application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and graphic processing units (GPUs) have been employed to improve the throughput of CNNs. More precisely, FPGAs have been recently adopted for accelerating the implementation of deep learning networks due to their ability to maximize parallelism as well as due to their energy efficiency. In this paper, we review recent existing techniques for accelerating deep learning networks on FPGAs. We highlight the key features employed by the various techniques for improving the acceleration performance. In addition, we provide recommendations for enhancing the utilization of FPGAs for CNNs acceleration. The techniques investigated in this paper represent the recent trends in FPGA-based accelerators of deep learning networks. Thus, this review is expected to direct the future advances on efficient hardware accelerators and to be useful for deep learning researchers.

This article has been accepted for publication in IEEE Access (December, 2018)

References in corpus (22)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• TensorFlow: A system for large-scale machine learning
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
• Recurrent Neural Network Regularization
• Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
• Going Deeper with Convolutions
• Convolutional Neural Networks for Sentence Classification
• FINN: A Framework for Fast, Scalable Binarized Neural Network Inference
• cuDNN: Efficient Primitives for Deep Learning
• Predicting Parameters in Deep Learning
• PlaNet - Photo Geolocation with Convolutional Neural Networks
• Hardware for Machine Learning: Challenges and Opportunities
• Bitwise Neural Networks
• Deep Learning on FPGAs: Past, Present, and Future
• Resiliency of Deep Neural Networks under Quantization
• EIE: Efficient Inference Engine on Compressed Deep Neural Network
• Metaheuristic Algorithms for Convolution Neural Network

Cited by in corpus (27)

• Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors
• Fast convolutional neural networks on FPGAs with hls4ml
• Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
• Semantics for Robotic Mapping, Perception and Interaction: A Survey
• Taurus: A Data Plane Architecture for Per-Packet ML
• 6G White Paper on Machine Learning in Wireless Communication Networks
• Autoencoders on FPGAs for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider
• ShEF: Shielded Enclaves for Cloud FPGAs
• Charged particle tracking via edge-classifying interaction networks
• CFU Playground: Full-Stack Open-Source Framework for Tiny Machine Learning (tinyML) Acceleration on FPGAs
• Event-based vision on FPGAs -- a survey
• Near-Precise Parameter Approximation for Multiple Multiplications on A Single DSP Block
• Convolutional Neural Networks for Speech Controlled Prosthetic Hands
• FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation
• Optimization of FPGA-based CNN Accelerators Using Metaheuristics
• Neural Network Methods for Radiation Detectors and Imaging
• Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-Processors
• Single-Shot Matrix-Matrix Multiplication Optical Tensor Processor for Deep Learning
• Variational Quantum Support Vector Machine based on $\Gamma$ matrix expansion and Variational Universal-Quantum-State Generator
• Stress-induced Artificial neuron spiking in Diffusive memristors
• Duet: Creating Harmony between Processors and Embedded FPGAs
• Real-time Artificial Intelligence for Accelerator Control: A Study at the Fermilab Booster
• REX: Revisiting Budgeted Training with an Improved Schedule
• From the Pursuit of Universal AGI Architecture to Systematic Approach to Heterogenous AGI: Addressing Alignment, Energy, & AGI Grand Challenges
• When Side-Channel Attacks Break the Black-Box Property of Embedded Artificial Intelligence
• Spatiotemporal Ground Reaction Force Analysis using Convolutional Neural Networks to Analyze Parkinsonian Gait
• A Flexible FPGA-Based ISA Configurable SoC platform