Compress Then Adapt? No, Do It Together via Task-aware Union of Subspaces

Jingze Ge, Yun Liu, Xue Geng +4

Adapting large pretrained models to diverse tasks is now routine, yet the two dominant strategies of parameter-efficient fine-tuning (PEFT) and low-rank compression are typically c…