6 citations · 13 across the 5 of their papers we have counts for
5 papers
Riemannian Natural Gradient Methods
Jiang Hu, Ruicheng Ao, Anthony Man-Cho So +2
This paper studies large-scale optimization problems on Riemannian manifolds whose objective function is a finite sum of negative log-probability losses. Such problems arise in var…
NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep Learning
Minghan Yang, Dong Xu, Qiwen Cui +2
In this paper, a novel second-order method called NG+ is proposed. By following the rule ``the shape of the gradient equals the shape of the parameter", we define a generalized fis…
Enhance Curvature Information by Structured Stochastic Quasi-Newton Methods
Minghan Yang, Dong Xu, Hongyu Chen +2
In this paper, we consider stochastic second-order methods for minimizing a finite summation of nonconvex functions. One important key is to find an ingenious but cheap scheme to i…
Sketchy Empirical Natural Gradient Methods for Deep Learning
Minghan Yang, Dong Xu, Zaiwen Wen +2
In this paper, we develop an efficient sketchy empirical natural gradient method (SENG) for large-scale deep learning problems. The empirical Fisher information matrix is usually l…
A Stochastic Extra-Step Quasi-Newton Method for Nonsmooth Nonconvex Optimization
Minghan Yang, Andre Milzarek, Zaiwen Wen +1
In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems. We assume that the gradient o…