MoRe Fine-Tuning with 10x Fewer Parameters

Wenxuan Tan, Nicholas Roberts, Tzu-Heng Huang +5

Parameter-efficient fine-tuning (PEFT) techniques have unlocked the potential to cheaply and easily specialize large pretrained models. However, the most prominent approaches, like…