On the Natural Gradient of the Evidence Lower Bound

Nihat Ay, Jesse van Oostrum, Adwait Datar

This article studies the Fisher-Rao gradient, also referred to as the natural gradient, of the evidence lower bound (ELBO) which plays a central role in generative machine learning…