Convergence guarantees for Muon: New parameter regimes and generalizations

Arthur C. B. de Oliveira, Dhruv D. Jatkar, Guilherme S. Vicinansa +1

In this paper, we establish the first asymptotic convergence guarantees for the Muon algorithm through a more accurate proxy for the Newton-Schultz iteration than the typical matri…