2 papers
cs.LG2025
Muon Optimizes Under Spectral Norm Constraints
Lizhang Chen, Jonathan Li, Qiang Liu
The pursuit of faster optimization algorithms remains an active and important research direction in deep learning. Recently, the Muon optimizer [JJB+24] has demonstrated promising…
cs.LG2025
Improving Adaptive Moment Optimization via Preconditioner Diagonalization
Son Nguyen, Bo Liu, Lizhang Chen +1
Modern adaptive optimization methods, such as Adam and its variants, have emerged as the most widely used tools in deep learning over recent years. These algorithms offer automatic…