Catastrophic Forgetting is Low-Rank: A Function-Space Theory for Continual Adaptation

Ido Nitzan Hidekel, Dan Raviv

Catastrophic forgetting in continual adaptation is usually studied through parameter drift, replay, or distillation, but these views do not identify which output-space directions a…