SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation

Yixian Shen, Qi Bi, Jia-Hong Huang +3

Low-rank adaptation (LoRA) has been demonstrated effective in reducing the trainable parameter number when fine-tuning a large foundation model (LLM). However, it still encounters…