185 citations · 295 across the 17 of their papers we have counts for
3 papers · 1 filter
Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation
Robert M. Gower, Guillaume Garrigos, Nicolas Loizou +3
We provide a general convergence theorem of an idealized stochastic Polyak step size called SPS. Besides convexity, we only assume a local expected gradient bound, that include…
Function Value Learning: Adaptive Learning Rates Based on the Polyak Stepsize and Function Splitting in ERM
Guillaume Garrigos, Robert M. Gower, Fabian Schaipp
Here we develop variants of SGD (stochastic gradient descent) with an adaptive step size that make use of the sampled loss values. In particular, we focus on solving a finite sum-o…
Provable convergence guarantees for black-box variational inference
Justin Domke, Guillaume Garrigos, Robert Gower
Black-box variational inference is widely used in situations where there is no proof that its stochastic optimization succeeds. We suggest this is due to a theoretical gap in exist…