Scalable Second Order Optimization for Deep Learning

Rohan Anil, Vineet Gupta, Tomer Koren +2

Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent. Second-order optimizatio…