1 citations · 2 across the 4 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
MeqMuon: Matrix-Equilibrating Muon for LLM Pretraining
Chang-Wei Shi, Xu Wang, Wu-Jun Li
The success of large language models (LLMs) has been accompanied by continued growth in model size and pretraining costs. Muon offers high accuracy and training efficiency in LLM p…
cs.LG2026
Ordered Local Momentum for Asynchronous Distributed Learning under Arbitrary Delays
Chang-Wei Shi, Shi-Shang Wang, Wu-Jun Li
Momentum SGD (MSGD) serves as a foundational optimizer in training deep models due to momentum's key role in accelerating convergence and enhancing generalization. Meanwhile, async…