most citedMPipeMoE: Memory Efficient MoE for Pre-trained Models with Adaptive Pipeline Parallelism

10 citations · 10 across the 1 of their papers we have counts for

collaborators

1 paper

cs.DC2025★ 10 cited

MPipeMoE: Memory Efficient MoE for Pre-trained Models with Adaptive Pipeline Parallelism

Zheng Zhang, Donglin Yang, Yaqi Xia +4

Recently, Mixture-of-Experts (MoE) has become one of the most popular techniques to scale pre-trained models to extraordinarily large sizes. Dynamic activation of experts allows fo…