3 papers
cs.LG2026
MuonIO: Principled Norm-Aware Descent for Embedding Tables and Language Model Heads
Linkai Ma, Xinyu Luo, Mengbo Wang +3
The Muon optimizer derives its update rule for hidden linear layers by solving a local linearization of the loss penalized by the spectral norm, motivated by an RMS-stability argum…
cs.CL2025
Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner
Bolian Li, Yanran Wu, Xinyu Luo +1
Aligning large language models (LLMs) with human preferences has become a critical step in their development. Recent research has increasingly focused on test-time alignment, where…
cs.LG2025
Stacey: Promoting Stochastic Steepest Descent via Accelerated -Smooth Nonconvex Optimization
Xinyu Luo, Cedar Site Bai, Bolian Li +3
While popular optimization methods such as SGD, AdamW, and Lion depend on steepest descent updates in either or norms, there remains a critical gap in handli…