Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
What Makes Recurrence Effective in Looped Language Models?
Xinlin Zhuang, Siyuan Wang, Imran Razzak +1
Looped language models (LoopLMs) increase computational depth through parameter sharing, offering a path to scale inference computation without adding parameters. However, it remai…
cs.LG2026
Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering
Xianliang Li, Zihan Zhang, Weiyang Liu +1
Muon has recently demonstrated strong empirical performance in large language model training, but the theoretical role of momentum in Muon remains unclear. Existing analyses of Muo…
cs.LG2026
AMO: Operator-level Adaptive Muon Orthogonalization
Xinlin Zhuang, Panyi Ouyang, Yichen Li +7
Muon has recently emerged as a competitive alternative to AdamW for large-scale pre-training, with orthogonalization via Newton-Schulz (NS) iteration as its core operation. Standar…