3 citations · 3 across the 12 of their papers we have counts for
Showing 2025Show all
3 papers · 1 filter
cs.LG2025
Cautious Weight Decay
Lizhang Chen, Jonathan Li, Kaizhao Liang +6
We introduce Cautious Weight Decay (CWD), a one-line, optimizer-agnostic modification that applies weight decay only to parameter coordinates whose signs align with the optimizer u…
cs.LG2025
Muon Optimizes Under Spectral Norm Constraints
Lizhang Chen, Jonathan Li, Qiang Liu
The pursuit of faster optimization algorithms remains an active and important research direction in deep learning. Recently, the Muon optimizer [JJB+24] has demonstrated promising…
cs.LG2025
Improving Adaptive Moment Optimization via Preconditioner Diagonalization
Son Nguyen, Bo Liu, Lizhang Chen +1
Modern adaptive optimization methods, such as Adam and its variants, have emerged as the most widely used tools in deep learning over recent years. These algorithms offer automatic…