DMuon: Efficient Distributed Muon Training with Near-Adam Overhead

Vincent Chen, Starrick Liu, Regis Cheng +8

Matrix-orthogonalization-based optimizers, exemplified by Muon, have demonstrated strong convergence behavior across a wide range of modern deep learning workloads. The matrix-awar…