Spectral Imbalance Causes Forgetting in Low-Rank Continual Adaptation

Hao Gu, Mao-Lin Luo, Zi-Hao Zhou +3

Parameter-efficient continual learning aims to adapt pre-trained models to sequential tasks without forgetting previously acquired knowledge. Most existing approaches treat continu…