MoE${}^2$: Optimizing Collaborative Inference for Edge Large Language Models

Lyudong Jin, Yanning Zhang, Yanhan Li +4

Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of natural language processing tasks. Exploiting the heterogeneous capabilities of edge L…