AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models

Arash Lagzian, Paniz Halvachi, Junming Zhang +2

Muon improves large-scale training by applying a spectral-norm steepest-descent update to matrix parameters, but practical models also contain parameter blocks that do not fit dens…