When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning

Tanguy Dieudonné, Giulia Lanzillotta, Enis Simsar +2

Low-Rank Adaptation (LoRA) has become the standard tool for parameter-efficient fine-tuning of large pretrained models. When applied sequentially across tasks in Continual Learning…