A Formalization of The Natural Gradient Method for General Similarity Measures

Anton Mallasto, Tom Dela Haije, Aasa Feragen

In optimization, the natural gradient method is well-known for likelihood maximization. The method uses the Kullback-Leibler divergence, corresponding infinitesimally to the Fisher…