Muon Optimizer Accelerates Grokking

Amund Tveit, Bjørn Remseth, Arve Skogvold

This paper investigates the impact of different optimizers on the grokking phenomenon, where models exhibit delayed generalization. We conducted experiments across seven numerical…