TuckA: Hierarchical Compact Tensor Experts for Efficient Fine-Tuning

Qifeng Lei, Zhiyong Yang, Qianqian Xu +3

Efficiently fine-tuning pre-trained models for downstream tasks is a key challenge in the era of foundation models. Parameter-efficient fine-tuning (PEFT) presents a promising solu…