AdaGrad does not adapt to Hölder-smoothness for composite objectives

Matia Bojovic, Saverio Salzo, Massimiliano Pontil

We exhibit a simple deterministic one-dimensional convex composite optimization problem for which AdaGrad scheme does not achieve the classical convergence rate $\mathcal{O}(n^{-(1…