2 papers
cs.AI2025
Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gate
Zhiyuan Fang, Zicong Hong, Yuegui Huang +5
Large Language Models (LLMs) have demonstrated impressive performance across various tasks, and their application in edge scenarios has attracted significant attention. However, sp…
cs.LG2025
Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline
Zhiyuan Fang, Yuegui Huang, Zicong Hong +5
Mixture of Experts (MoE), with its distinctive sparse structure, enables the scaling of language models up to trillions of parameters without significantly increasing computational…