Muon Can Outperform Dedicated Continual Learning Methods

Sebastian George Sincari, Bogdan Alexandru Gheorghe, Antonio Barbalau

Continual learning with Low-Rank Adapters (LoRA) typically mitigates forgetting by penalizing the overlap between a new update and the accumulated past weights, which discourages c…