Improving performance of recurrent neural network with relu nonlinearity

Sachin S. Talathi, Aniket Vartak

In recent years significant progress has been made in successfully training recurrent neural networks (RNNs) on sequence learning problems involving long range temporal dependencie…