1 citations · 2 across the 13 of their papers we have counts for
3 papers · 1 filter
Drop-Muon: Update Less, Converge Faster
Kaja Gruntkowska, Yassine Maziane, Zheng Qu +1
Conventional wisdom in deep learning optimization dictates updating all layers at every step-a principle followed by all recent state-of-the-art optimizers such as Muon. In this wo…
Error Feedback for Muon and Friends
Kaja Gruntkowska, Alexander Gaponov, Zhirayr Tovmasyan +1
Recent optimizers like Muon, Scion, and Gluon have pushed the frontier of large-scale deep learning by exploiting layer-wise linear minimization oracles (LMOs) over non-Euclidean n…
Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)
Artem Riabinin, Egor Shulgin, Kaja Gruntkowska +1
Recent developments in deep learning optimization have brought about radically new algorithms based on the Linear Minimization Oracle (LMO) framework, such as and $\sf S…