Showing 2021Show all
2 papers · 1 filter
cs.LG2021
Coordinate descent on the orthogonal group for recurrent neural network training
Estelle Massart, Vinayak Abrol
We propose to use stochastic Riemannian coordinate descent on the orthogonal group for recurrent neural network training. The algorithm rotates successively two columns of the recu…
cs.LG2021
Activation function design for deep networks: linearity and effective initialisation
Michael Murray, Vinayak Abrol, Jared Tanner
The activation function deployed in a deep neural network has great influence on the performance of the network at initialisation, which in turn has implications for training. In t…