Sparse Low-rank Adaptation of Pre-trained Language Models

Ning Ding, Xingtai Lv, Qiaosen Wang +4

Fine-tuning pre-trained large language models in a parameter-efficient manner is widely studied for its effectiveness and efficiency. The popular method of low-rank adaptation (LoR…