Monkey Jump : MoE-Style PEFT for Efficient Multi-Task Learning

Nusrat Jahan Prottasha, Md Kowsher, Chun-Nam Yu +2

Mixture-of-experts variants of parameter-efficient fine-tuning enable per-token specialization, but they introduce additional trainable routers and expert parameters, increasing me…