GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection

Kai Yao, Zhenghan Song, Kaixin Wu +5

Parameter-Efficient Fine-Tuning (PEFT) has become a key strategy for adapting large language models, with recent advances in sparse tuning reducing overhead by selectively updating…