Curvature-Informed SGD via General Purpose Lie-Group Preconditioners

Omead Pooladzandi, Xi-Lin Li

We present a novel approach to accelerate stochastic gradient descent (SGD) by utilizing curvature information obtained from Hessian-vector products or finite differences of parame…