S2FT: Parameter-Efficient Fine-Tuning in Sparse Spectrum Domain

Baoquan Zhang, Zhehao Yu, Lisai Zhang +5

Parameter Efficient Fine-Tuning (PEFT) is a key technique for adapting a large pretrained model to downstream tasks by fine-tuning only a small number of parameters. Recent methods…