D${}^2$MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving

Haodong Wang, Qihua Zhou, Zicong Hong +1

The mixture of experts (MoE) model is a sparse variant of large language models (LLMs), designed to hold a better balance between intelligent capability and computational overhead.…