Disentangling Task Conflicts in Multi-Task LoRA via Orthogonal Gradient Projection

Ziyu Yang, Guibin Chen, Yuxin Yang +2

Multi-Task Learning (MTL) combined with Low-Rank Adaptation (LoRA) has emerged as a promising direction for parameter-efficient deployment of Large Language Models (LLMs). By shari…