1 paper · 1 filter
Vishrut Goyal, Rohan Ramkumar
The Muon optimizer orthogonalizes each update matrix, setting all of its singular values to one, and has proven highly effective for training large language models. It remains uncl…