Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
MuonIO: Principled Norm-Aware Descent for Embedding Tables and Language Model Heads
Linkai Ma, Xinyu Luo, Mengbo Wang +3
The Muon optimizer derives its update rule for hidden linear layers by solving a local linearization of the loss penalized by the spectral norm, motivated by an RMS-stability argum…
cs.LG2025
Stacey: Promoting Stochastic Steepest Descent via Accelerated -Smooth Nonconvex Optimization
Xinyu Luo, Cedar Site Bai, Bolian Li +3
While popular optimization methods such as SGD, AdamW, and Lion depend on steepest descent updates in either or norms, there remains a critical gap in handli…