Time-Scale Coupling Between States and Parameters in Recurrent Neural Networks

Lorenzo Livi

We show that gating mechanisms in recurrent neural networks (RNNs) induce lag-dependent and direction-dependent effective learning rates, even when training uses a fixed, global st…