Efficient Numerical Algorithm for Large-Scale Damped Natural Gradient Descent

Yixiao Chen, Hao Xie, Han Wang

We propose a new algorithm for efficiently solving the damped Fisher matrix in large-scale scenarios where the number of parameters significantly exceeds the number of available sa…