Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator

Zhishuai Guo, Yi Xu, Wotao Yin +2

Although adaptive optimization algorithms have been successful in many applications, there are still some mysteries in terms of convergence analysis that have not been unraveled. T…