Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning

Joana Pasquali, Ramiro N. Barros, Arthur S. Bianchessi +7

LoRA is widely adopted for continual fine-tuning of Large Language Models due to its parameter efficiency, modularity across tasks, and compatibility with replay strategies. Howeve…