Convergence of ADAM for Lipschitz Objective Functions

Abstract

The aim of this paper is to prove the exponential convergence, local and global, of Adam algorithm under precise conditions on the parameters, when the objective function lacks differentiability. More precisely, we require Lipschitz continuity, and control on the gradient whenever it exists. We provide also examples of interesting functions that satisfies the required restrictions.