Muon-OGD: Muon-based Spectral Orthogonal Gradient Projection for LLM Continual Learning

Binghang Lu, Zheyuan Deng, Runyu Zhang +6

A central challenge in continual learning for large language models (LLMs) is catastrophic forgetting, where adapting to new tasks can substantially degrade performance on previous…