Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models

Xinxin Liu, Aaron Thomas, Cheng Zhang +3

Parameter-Efficient Fine-Tuning (PEFT) has gained prominence through low-rank adaptation methods like LoRA. In this paper, we focus on sparsity-based PEFT (SPEFT), which introduces…