DizzyRNN: Reparameterizing Recurrent Neural Networks for Norm-Preserving Backpropagation

Victor Dorobantu, Per Andre Stromhaug, Jess Renteria

The vanishing and exploding gradient problems are well-studied obstacles that make it difficult for recurrent neural networks to learn long-term time dependencies. We propose a rep…