Routing Experts: Learning to Route Dynamic Experts in Multi-modal Large Language Models

Qiong Wu, Zhaoxi Ke, Yiyi Zhou +2

Recently, mixture of experts (MoE) has become a popular paradigm for achieving the trade-off between modal capacity and efficiency of multi-modal large language models (MLLMs). Dif…