Coordinate descent on the orthogonal group for recurrent neural network training

Estelle Massart, Vinayak Abrol

We propose to use stochastic Riemannian coordinate descent on the orthogonal group for recurrent neural network training. The algorithm rotates successively two columns of the recu…