Dynamic Regret of Adaptive Gradient Methods for Strongly Convex Problems

Parvin Nazari, Esmaile Khorram

Adaptive gradient algorithms such as ADAGRAD and its variants have gained popularity in the training of deep neural networks. While many works as for adaptive methods have focused…