LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluation

Hongyun Zhou, Xiangyu Lu, Wang Xu +3

Low-Rank Adaptation (LoRA) is currently the most commonly used Parameter-efficient fine-tuning (PEFT) method, it introduces auxiliary parameters for each layer to fine-tune the pre…