PaLoRA: Paced Low-Rank Adaptation for Continual Learning

Yuxuan Li, Fanhu Zeng, Hao Tang

LoRA-based continual learning methods mitigate catastrophic forgetting through various mechanisms, yet nearly all complement these with small learning rates as a heuristic to restr…