1 citations · 2 across the 13 of their papers we have counts for
Showing 2026Show all
3 papers · 1 filter
cs.LG2026
Towards joint scaling laws with optimal batch size schedules
Jiaxiang Li, Zhiqi Bu, Shiyun Xu
Modern deep learning typically keeps the batch size static throughout training, thus overlooking the joint effect of learning rate and batch size on the training dynamics. In this…
cs.LG2026
Why Muon Outperforms Adam: A Curvature Perspective
Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2
Muon improves training efficiency over Adam in large language-model training by about two times, but the local geometric source of this advantage remains unclear. Our work takes a…
cs.LG2026
Demystifying Manifold Constraints in LLM Pre-training
Kang An, Jiaxiang Li, Donald Goldfarb +1
The recent success of matrix optimizers (e.g., Muon) suggests that specific normalization of momentum, such as orthogonalization and row-wise normalization, benefits both the stabi…