5 citations · 5 across the 7 of their papers we have counts for
6 papers · 1 filter
Muon Learns More Robust and Transferable Features than Adam
Tianyu Ruan, Fengzhuo Zhang, Shuche Wang +1
Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs) and vision classifiers. Despite its efficiency advantage over Adam and SGD, t…
Why Muon Outperforms Adam: A Curvature Perspective
Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2
Muon improves training efficiency over Adam in large language-model training by about two times, but the local geometric source of this advantage remains unclear. Our work takes a…
Online Learning with Gradient-Variation Interval Regret
Yan-Feng Xie, Shuche Wang, Peng Zhao +1
This paper investigates non-stationary online learning using the metric of interval regret, which requires an online algorithm to perform well over every time interval. We propose…
Bandit Convex Optimization with Gradient Prediction Adaptivity
Shuche Wang, Adarsh Barik, Vincent Y. F. Tan
Bandit convex optimization (BCO) is a fundamental online learning framework with partial feedback, where the learner observes only the loss incurred at the chosen decision point in…
Parameter-free Algorithms for the Stochastically Extended Adversarial Model
Shuche Wang, Adarsh Barik, Peng Zhao +1
We develop the first parameter-free algorithms for the Stochastically Extended Adversarial (SEA) model, a framework that bridges adversarial and stochastic online convex optimizati…
Muon Outperforms Adam in Tail-End Associative Memory Learning
Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +6
The Muon optimizer is consistently faster than Adam in training Large Language Models (LLMs), yet the mechanism underlying its success remains unclear. This paper demystifies this…