1 citations · 1 across the 3 of their papers we have counts for
4 papers
CIPHER-MoE: Balancing Efficiency and Routing Fidelity in Trillion-Scale MoE Training
Jing Li, Jian Meng, Yingmeng Gao +8
Mixture-of-Experts (MoE) has been widely adopted in recent large language model (LLM) architectures. However, scaling up MoE in LLM training introduces system-level challenges on t…
Expert-Token Resonance MoE: Bidirectional Routing with Efficiency Affinity-Driven Active Selection
Jing Li, Zhijie Sun, Dachao Lin +5
Mixture-of-Experts (MoE) architectures enable efficient scaling of large language models by activating only a subset of parameters per input. However, existing MoE models suffer fr…
WindGP: Efficient Graph Partitioning on Heterogenous Machines
Li Zeng, Haohan Huang, Binfan Zheng +6
Graph Partitioning is widely used in many real-world applications such as fraud detection and social network analysis, in order to enable the distributed graph computing on large g…
LocMoE: A Low-Overhead MoE for Large Language Model Training
Jing Li, Zhijie Sun, Xuan He +6
The Mixtures-of-Experts (MoE) model is a widespread distributed and integrated learning method for large language models (LLM), which is favored due to its ability to sparsify and…