13 citations · 18 across the 20 of their papers we have counts for
Showing 2026Show all
3 papers · 1 filter
math.OC2026
Phases of Muon: When Muon Eclipses SignSGD
Elliot Paquette, Noah Marshall, Lucas Benigni +3
Recently, Muon and related spectral optimizers have demonstrated strong empirical performance as scalable stochastic methods, often outperforming Adam. Yet their behaviour remains…
cs.LG2026
High dimensional theory of two-phase optimizers
Atish Agarwala
The trend towards larger training setups has brought a renewed interest in partially asynchronous two-phase optimizers which optimize locally and then synchronize across workers. A…
cs.LG2026
What do near-optimal learning rate schedules look like?
Hiroki Naganuma, Atish Agarwala, Priya Kasimbeg +1
A basic unanswered question in neural network training is: what is the best learning rate schedule shape for a given workload? The choice of learning rate schedule is a key factor…