Recurrent neural network training with preconditioned stochastic gradient descent

Xi-Lin Li

This paper studies the performance of a recently proposed preconditioned stochastic gradient descent (PSGD) algorithm on recurrent neural network (RNN) training. PSGD adaptively es…