5 citations · 6 across the 3 of their papers we have counts for
3 papers
math.OC2021★ 1 cited
NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep Learning
Minghan Yang, Dong Xu, Qiwen Cui +2
In this paper, a novel second-order method called NG+ is proposed. By following the rule ``the shape of the gradient equals the shape of the parameter", we define a generalized fis…
math.OC2020
Enhance Curvature Information by Structured Stochastic Quasi-Newton Methods
Minghan Yang, Dong Xu, Hongyu Chen +2
In this paper, we consider stochastic second-order methods for minimizing a finite summation of nonconvex functions. One important key is to find an ingenious but cheap scheme to i…
math.OC2020★ 5 cited
Sketchy Empirical Natural Gradient Methods for Deep Learning
Minghan Yang, Dong Xu, Zaiwen Wen +2
In this paper, we develop an efficient sketchy empirical natural gradient method (SENG) for large-scale deep learning problems. The empirical Fisher information matrix is usually l…