39 citations · 41 across the 7 of their papers we have counts for
4 papers · 1 filter
Kronecker-factored Quasi-Newton Methods for Deep Learning
Yi Ren, Achraf Bahamou, Donald Goldfarb
Second-order methods have the capability of accelerating optimization by using much richer curvature information than first-order methods. However, most are impractical for deep le…
Practical Quasi-Newton Methods for Training Deep Neural Networks
Donald Goldfarb, Yi Ren, Achraf Bahamou
We consider the development of practical stochastic quasi-Newton, and in particular Kronecker-factored block-diagonal BFGS and L-BFGS methods, for training deep neural networks (DN…
Stochastic Flows and Geometric Optimization on the Orthogonal Group
Krzysztof Choromanski, David Cheikhi, Jared Davis +12
We present a new class of stochastic, geometrically-driven optimization algorithms on the orthogonal group and naturally reductive homogeneous manifolds obtained from the ac…
A Dynamic Sampling Adaptive-SGD Method for Machine Learning
Achraf Bahamou, Donald Goldfarb
We propose a stochastic optimization method for minimizing loss functions, expressed as an expected value, that adaptively controls the batch size used in the computation of gradie…