LoRA-Based Continual Learning with Constraints on Critical Parameter Changes

Shimou Ling, Liang Zhang, Jiangwei Zhao +2

LoRA-based continual learning represents a promising avenue for leveraging pre-trained models in downstream continual learning tasks. Recent studies have shown that orthogonal LoRA…