Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
Taking the Road Less Scheduled with Adaptive Polyak Steps
Dimitris Oikonomou, Matthew Buchholz, Yuen-Man Pun +2
Schedule-Free SGD, proposed in [Defazio et al., 2024], achieves optimal convergence rates without requiring the training horizon in advance, by replacing learning rate schedules wi…
cs.LG2025
Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation
Robert M. Gower, Guillaume Garrigos, Nicolas Loizou +3
We provide a general convergence theorem of an idealized stochastic Polyak step size called SPS. Besides convexity, we only assume a local expected gradient bound, that include…