22 citations · 35 across the 7 of their papers we have counts for
Showing 2023Show all
2 papers · 1 filter
cs.CL2023★ 2 cited
Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation
Chen Dun, Mirian Hipolito Garcia, Guoqing Zheng +3
Large Language Models (LLMs) have the ability to solve a variety of tasks, such as text summarization and mathematical questions, just out of the box, but they are often trained wi…
cs.LG2023★ 1 cited
Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings
Yehya Farhat, Hamza ElMokhtar Shili, Fangshuo Liao +7
Mixture-of-Experts (MoEs) achieve scalability by dynamically activating subsets of their components. Yet, understanding how expertise emerges through joint training of gating mecha…