Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation

Robert M. Gower, Guillaume Garrigos, Nicolas Loizou +3

We provide a general convergence theorem of an idealized stochastic Polyak step size called SPS${}^{\ast }$. Besides convexity, we only assume a local expected gradient bound, that include…