Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory

Kyle Helfrich, Qiang Ye

Several variants of recurrent neural networks (RNNs) with orthogonal or unitary recurrent matrices have recently been developed to mitigate the vanishing/exploding gradient problem…