Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics

Mark Rhee, Jamie Simon, Dhruva Karkada

Matrix factorization (i.e., problems of the form ${\min }_{\mathbf{P},\mathbf{Q}}\Vert {\mathbf{M}}^{\star }-{\mathbf{P}}^{\top }\mathbf{Q}{\Vert }_{\mathrm{F}}^2$) is a minimal learning problem that ex…