Krylov Subspace Descent for Deep Learning

Oriol Vinyals, Daniel Povey

In this paper, we propose a second order optimization method to learn models where both the dimensionality of the parameter space and the number of training samples is high. In our…