Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering

Xianliang Li, Zihan Zhang, Weiyang Liu +1

Muon has recently demonstrated strong empirical performance in large language model training, but the theoretical role of momentum in Muon remains unclear. Existing analyses of Muo…