AMO: Adaptive Muon Orthogonalization

Xinlin Zhuang, Panyi Ouyang, Yichen Li +7

Muon has recently emerged as a competitive alternative to AdamW for large-scale pre-training, with orthogonalization via Newton-Schulz (NS) iterations as its core operation. Existi…