Continual Gradient Low-Rank Projection Fine-Tuning for LLMs

Chenxu Wang, Yilin Lyu, Zicheng Sun +1

Continual fine-tuning of Large Language Models (LLMs) is hampered by the trade-off between efficiency and expressiveness. Low-Rank Adaptation (LoRA) offers efficiency but constrain…