21 citations · 22 across the 3 of their papers we have counts for
4 papers
AsymptoticNG: A regularized natural gradient optimization algorithm with look-ahead strategy
Zedong Tang, Fenlong Jiang, Junke Song +5
Optimizers that further adjust the scale of gradient, such as Adam, Natural Gradient (NG), etc., despite widely concerned and used by the community, are often found poor generaliza…
A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations
Jianfeng Lu, Yulong Lu, Min Wang
This paper concerns the a priori generalization analysis of the Deep Ritz Method (DRM) [W. E and B. Yu, 2017], a popular neural-network-based method for solving high dimensional pa…
Eigenvalue-corrected Natural Gradient Based on a New Approximation
Kai-Xin Gao, Xiao-Lei Liu, Zheng-Hai Huang +5
Using second-order optimization methods for training deep neural networks (DNNs) has attracted many researchers. A recently proposed method, Eigenvalue-corrected Kronecker Factoriz…
A Trace-restricted Kronecker-Factored Approximation to Natural Gradient
Kai-Xin Gao, Xiao-Lei Liu, Zheng-Hai Huang +4
Second-order optimization methods have the ability to accelerate convergence by modifying the gradient through the curvature matrix. There have been many attempts to use second-ord…