Monarch-Muon: Structure-Aware Muon for Fast and Memory-Efficient Pretraining

Alexander Miasnikov, Ruslan Kabirov, Gleb Molodtsov +7

Training large language models is constrained by both wall-clock time and memory footprint, and recent matrix-aware optimizers sharpen this tradeoff: Muon can improve optimization…