Symmetric Rank-One Quasi-Newton Methods for Deep Learning Using Cubic Regularization

Aditya Ranganath, Mukesh Singhal, Roummel Marcia

Stochastic gradient descent and other first-order variants, such as Adam and AdaGrad, are commonly used in the field of deep learning due to their computational efficiency and low-…