Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gate

Zhiyuan Fang, Zicong Hong, Yuegui Huang +5

Large Language Models (LLMs) have demonstrated impressive performance across various tasks, and their application in edge scenarios has attracted significant attention. However, sp…