CacheMuon: Using Temporal Preconditioning To Approximate Polar Factor

Bishnu Dev, Sushil Bohara, Martin Takáč +1

Muon is an optimizer that computes updates using the polar factor of the momentum matrix and has shown strong empirical performance across a range of training settings. A key compo…