Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
MuonIO: Principled Norm-Aware Descent for Embedding Tables and Language Model Heads
Linkai Ma, Xinyu Luo, Mengbo Wang +3
The Muon optimizer derives its update rule for hidden linear layers by solving a local linearization of the loss penalized by the spectral norm, motivated by an RMS-stability argum…
cs.LG2026
The Advantages of Fresh Sketching for Ridge Regression
Linkai Ma, Qilin Li, Petros Drineas
Over the past 25 years, sketching and sampling have become widely used tools for accelerating large-scale regression. In iterative randomized solvers, a basic design choice is whet…
cs.LG2024
Stochastic Rounding Implicitly Regularizes Tall-and-Thin Matrices
Gregory Dexter, Christos Boutsikas, Linkai Ma +2
Motivated by the popularity of stochastic rounding in the context of machine learning and the training of large-scale deep neural network models, we consider stochastic nearness ro…