Rank-Accuracy Trade-off for LoRA: A Gradient-Flow Analysis

Michael Rushka, Diego Klabjan

Previous empirical studies have shown that LoRA achieves accuracy comparable to full-parameter methods on downstream fine-tuning tasks, even for rank-1 updates. By contrast, the th…