CopRA: A Progressive LoRA Training Strategy

Zhan Zhuang, Xiequn Wang, Yulong Zhang +3

Low-Rank Adaptation (LoRA) is a parameter-efficient technique for rapidly fine-tuning foundation models. In standard LoRA training dynamics, models tend to quickly converge to a lo…