Training recurrent networks online without backtracking

Yann Ollivier, Corentin Tallec, Guillaume Charpiat

We introduce the "NoBackTrack" algorithm to train the parameters of dynamical systems such as recurrent neural networks. This algorithm works in an online, memoryless setting, thus…