Showing 2025Show all
3 papers · 1 filter
cs.LG2025
Taking the Road Less Scheduled with Adaptive Polyak Steps
Dimitris Oikonomou, Matthew Buchholz, Yuen-Man Pun +2
Schedule-Free SGD, proposed in [Defazio et al., 2024], achieves optimal convergence rates without requiring the training horizon in advance, by replacing learning rate schedules wi…
stat.ML2025
Fisher meets Feynman: score-based variational inference with a product of experts
Diana Cai, Robert M. Gower, David M. Blei +1
We introduce a highly expressive yet distinctly tractable family for black-box variational inference (BBVI). Each member of this family is a weighted product of experts (PoE), and…
cs.LG2025
An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants
Michael Crawshaw, Chirag Modi, Mingrui Liu +1
To define a steepest descent method over a neural network, we need to choose a norm for each layer, a way to aggregate these norms across layers, and whether to use normalization.…