Low-rank Attention Side-Tuning for Parameter-Efficient Fine-Tuning

Ningyuan Tang, Minghao Fu, Ke Zhu +1

In finetuning a large pretrained model to downstream tasks, parameter-efficient fine-tuning (PEFT) methods can effectively finetune pretrained models with few trainable parameters,…