MoE${}^2$-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation

Qingyu Yang, Haonan He, Minglei Li +4

Mixture-of-Experts (MoE) architectures have been widely adopted in large language models, yet parameter-efficient fine-tuning (PEFT) for MoE models remains underexplored. Existing…