Ada-K Routing: Boosting the Efficiency of MoE-based LLMs

Tongtian Yue, Longteng Guo, Jie Cheng +2

In the era of Large Language Models (LLMs), Mixture-of-Experts (MoE) architectures offer a promising approach to managing computational costs while scaling up model parameters. Con…