9 papers
Muon Learns More Robust and Transferable Features than Adam
Tianyu Ruan, Fengzhuo Zhang, Shuche Wang +1
Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs) and vision classifiers. Despite its efficiency advantage over Adam and SGD, t…
Why Muon Outperforms Adam: A Curvature Perspective
Shuche Wang, Fengzhuo Zhang, Jiaxiang Li +2
Muon improves training efficiency over Adam in large language-model training by about two times, but the local geometric source of this advantage remains unclear. Our work takes a…
Online Learning with Gradient-Variation Interval Regret
Yan-Feng Xie, Shuche Wang, Peng Zhao +1
This paper investigates non-stationary online learning using the metric of interval regret, which requires an online algorithm to perform well over every time interval. We propose…
List Reconstruction Problem with List Size Two
Binh Vu, Shuche Wang, Van Khu Vu
The problem of computing the cardinality of the intersection of multiple balls in the Hamming space has attracted a lot of attention recently due to their applications in the list…
Bandit Convex Optimization with Gradient Prediction Adaptivity
Shuche Wang, Adarsh Barik, Vincent Y. F. Tan
Bandit convex optimization (BCO) is a fundamental online learning framework with partial feedback, where the learner observes only the loss incurred at the chosen decision point in…
Parameter-free Algorithms for the Stochastically Extended Adversarial Model
Shuche Wang, Adarsh Barik, Peng Zhao +1
We develop the first parameter-free algorithms for the Stochastically Extended Adversarial (SEA) model, a framework that bridges adversarial and stochastic online convex optimizati…