351 citations · 835 across the 27 of their papers we have counts for
Showing stat.MLShow all
2 papers · 1 filter
stat.ML2020★ 4 cited
Convergence of adaptive algorithms for weakly convex constrained optimization
Ahmet Alacaoglu, Yura Malitsky, Volkan Cevher
We analyze the adaptive first order algorithm AMSGrad, for solving a constrained stochastic optimization problem with a weakly convex objective. We prove the $\mathcal{\tilde O}(t^…
stat.ML2020★ 5 cited
A new regret analysis for Adam-type algorithms
Ahmet Alacaoglu, Yura Malitsky, Panayotis Mertikopoulos +1
In this paper, we focus on a theory-practice gap for Adam and its variants (AMSgrad, AdamNC, etc.). In practice, these algorithms are used with a constant first-order moment parame…