On the Convergence Analysis of Muon

Wei Shen, Ruichuan Huang, Minhui Huang +2

The majority of parameters in neural networks are naturally represented as matrices. However, most commonly used optimizers treat these matrix parameters as flattened vectors durin…