Recurrent Neural Networks Hardware Implementation on FPGA

Abstract

Recurrent Neural Networks (RNNs) have the ability to retain memory and learn data sequences. Due to the recurrent nature of RNNs, it is sometimes hard to parallelize all its computations on conventional hardware. CPUs do not currently offer large parallelism, while GPUs offer limited parallelism due to sequential components of RNN models. In this paper we present a hardware implementation of Long-Short Term Memory (LSTM) recurrent network on the programmable logic Zynq 7020 FPGA from Xilinx. We implemented a RNN with $2$ layers and $128$ hidden units in hardware and it has been tested using a character level language model. The implementation is more than $21\times$ faster than the ARM CPU embedded on the Zynq 7020 FPGA. This work can potentially evolve to a RNN co-processor for future mobile devices.

7 pages, 8 figures, changed format, added figures, added references, modified introduction

References in corpus (5)

• Deep Learning in Neural Networks: An Overview
• Sequence to Sequence Learning with Neural Networks
• LSTM: A Search Space Odyssey
• Recurrent Neural Network Regularization
• Explain Images with Multimodal Recurrent Neural Networks

Cited by in corpus (4)

• Deep Learning-based Cattle Activity Classification Using Joint Time-frequency Data Representation
• ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA
• Machine Learning for Continuous Quantum Error Correction on Superconducting Qubits
• Serving Recurrent Neural Networks Efficiently with a Spatial Accelerator