A Dynamical Theory of LoRA in Continual Learning

Théo Marchetta, Filippo Alessandroni, Alessandro Breccia +2

Despite the widespread use of Low-Rank Adaptation (LoRA), little is known about its dynamics in continual learning and the mechanisms by which low-rank updates affect catastrophic…