3 papers
cs.LG2026
LiMuon: Light and Fast Muon Optimizer for Large Models
Feihu Huang, Yuning Luo, Songcan Chen
Large models recently are widely applied in machine learning, so efficient training of large models has received widespread attention. More recently, the useful Muon optimizer is s…
cs.LG2026
CLion: Efficient Cautious Lion Optimizer with Enhanced Generalization
Feihu Huang, Guanyi Zhang, Songcan Chen
Lion optimizer is a popular learning-based optimization algorithm in machine learning, which shows impressive performance in training many deep learning models. Although convergenc…
cs.LG2026
HomeAdam: Adam and AdamW Algorithms Sometimes Go Home to Obtain Better Provable Generalization
Feihu Huang, Guanyi Zhang, Songcan Chen
Adam and AdamW are a class of default optimizers for training deep learning models in machine learning. These adaptive algorithms converge faster but generalize worse compared to S…