Convergence Analysis of Real-time Recurrent Learning (RTRL) for a class of Recurrent Neural Networks

Samuel Chun-Hei Lam, Justin Sirignano, Konstantinos Spiliopoulos

Recurrent neural networks (RNNs) are commonly trained with the truncated backpropagation-through-time (TBPTT) algorithm. For the purposes of computational tractability, the TBPTT a…