Convergence of Online Adaptive and Recurrent Optimization Algorithms

Pierre-Yves Massé, Yann Ollivier

We prove local convergence of several notable gradient descent algorithms used in machine learning, for which standard stochastic gradient descent theory does not apply directly. T…