2 citations · 3 across the 2 of their papers we have counts for
3 papers
cs.LG2020
Eigenvalue-corrected Natural Gradient Based on a New Approximation
Kai-Xin Gao, Xiao-Lei Liu, Zheng-Hai Huang +5
Using second-order optimization methods for training deep neural networks (DNNs) has attracted many researchers. A recently proposed method, Eigenvalue-corrected Kronecker Factoriz…
cs.LG2020★ 1 cited
A Trace-restricted Kronecker-Factored Approximation to Natural Gradient
Kai-Xin Gao, Xiao-Lei Liu, Zheng-Hai Huang +4
Second-order optimization methods have the ability to accelerate convergence by modifying the gradient through the curvature matrix. There have been many attempts to use second-ord…
cs.LG2020★ 2 cited
EAdam Optimizer: How Impact Adam
Wei Yuan, Kai-Xin Gao
Many adaptive optimization methods have been proposed and used in deep learning, in which Adam is regarded as the default algorithm and widely used in many deep learning frameworks…