MoEGen: Mixture-of-Experts for Instance-Adaptive LoRA Generation

Yiming Zeng, Lei Lu, Zexin Li +9

Parameter-efficient fine-tuning (PEFT) enables efficient adaptation of large language models, but existing MoE-based PEFT methods typically improve capacity by storing multiple ful…