2 papers
math.OC2020
Enhance Curvature Information by Structured Stochastic Quasi-Newton Methods
Minghan Yang, Dong Xu, Hongyu Chen +2
In this paper, we consider stochastic second-order methods for minimizing a finite summation of nonconvex functions. One important key is to find an ingenious but cheap scheme to i…
math.OC2020
Sketchy Empirical Natural Gradient Methods for Deep Learning
Minghan Yang, Dong Xu, Zaiwen Wen +2
In this paper, we develop an efficient sketchy empirical natural gradient method (SENG) for large-scale deep learning problems. The empirical Fisher information matrix is usually l…