LoRA Dropout as a Sparsity Regularizer for Overfitting Control

Yang Lin, Xinyu Ma, Xu Chu +4

Parameter-efficient fine-tuning methods, represented by LoRA, play an essential role in adapting large-scale pre-trained models to downstream tasks. However, fine-tuning LoRA-serie…