Natural gradient descent with momentum

Anthony Nouy, Agustín Somacal

We consider the problem of approximating a function by an element of a nonlinear manifold which admits a differentiable parametrization, typical examples being neural networks with…