Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case

Meixuan He, Yuqing Liang, Jinlan Liu +1

Adam is a commonly used stochastic optimization algorithm in machine learning. However, its convergence is still not fully understood, especially in the non-convex setting. This pa…