Showing 2026Show all
3 papers · 1 filter
cs.LG2026
MuonIO: Principled Norm-Aware Descent for Embedding Tables and Language Model Heads
Linkai Ma, Xinyu Luo, Mengbo Wang +3
The Muon optimizer derives its update rule for hidden linear layers by solving a local linearization of the loss penalized by the spectral norm, motivated by an RMS-stability argum…
cs.LG2026
The Advantages of Fresh Sketching for Ridge Regression
Linkai Ma, Qilin Li, Petros Drineas
Over the past 25 years, sketching and sampling have become widely used tools for accelerating large-scale regression. In iterative randomized solvers, a basic design choice is whet…
math.NA2026
Stochastic Rounding Increases Small Singular Values
Linkai Ma, Tingzhou Yu, Petros Drineas
Over the past half-dozen years, stochastic rounding (SR) has regained significant attention as a quantization scheme for low-precision floating-point arithmetic, with applications…