A Gentle Tutorial of Recurrent Neural Network with Error Backpropagation

Abstract

We describe recurrent neural networks (RNNs), which have attracted great attention on sequential tasks, such as handwriting recognition, speech recognition and image to text. However, compared to general feedforward neural networks, RNNs have feedback loops, which makes it a little hard to understand the backpropagation step. Thus, we focus on basics, especially the error backpropagation to compute gradients with respect to model parameters. Further, we go into detail on how error backpropagation algorithm is applied on long short-term memory (LSTM) by unfolding the memory unit.

9 pages

References in corpus (1)

• Generalized K-fan Multimodal Deep Model with Shared Representations

Cited by in corpus (10)

• How to Build a Graph-Based Deep Learning Architecture in Traffic Domain: A Survey
• Recurrent Neural Networks (RNNs): A gentle Introduction and Overview
• Classification and Feature Transformation with Fuzzy Cognitive Maps
• Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors
• Video Action Understanding
• Sequence-to-Sequence Forecasting-aided State Estimation for Power Systems
• A Self-Correcting Deep Learning Approach to Predict Acute Conditions in Critical Care
• CNNs, LSTMs, and Attention Networks for Pathology Detection in Medical Data
• Teaching a Machine to Diagnose a Heart Disease; Beginning from digitizing scanned ECGs to detecting the Brugada Syndrome (BrS)
• Deep Learning and Open Set Malware Classification: A Survey