DT-LoRA: Reaching Beyond the Tangent Space of Low-Rank Adaptation

Yangming Zhang, Ling Yi, Datong Li +4

Low-rank adaptation (LoRA) often underperforms full fine-tuning. We investigate this performance gap through a tangent-space lens: factorized updates are confined to the tangent sp…