Showing 2025Show all
2 papers · 1 filter
cs.LG2025
Two Is Better Than One: Rotations Scale LoRAs
Hongcan Guo, Guoshun Nan, Yuan Yang +9
Scaling Low-Rank Adaptation (LoRA)-based Mixture-of-Experts (MoE) facilitates large language models (LLMs) to efficiently adapt to diverse tasks. However, traditional gating mechan…
cs.CL2025
Advancing Expert Specialization for Better MoE
Hongcan Guo, Haolang Lu, Guoshun Nan +8
Mixture-of-Experts (MoE) models enable efficient scaling of large language models (LLMs) by activating only a subset of experts per input. However, we observe that the commonly use…