activity
20182023
most citedFlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement

38 citations · 92 across the 5 of their papers we have counts for

collaborators
Showing 2021Show all

1 paper · 1 filter

cs.LG2021★ 7 cited

EvoMoE: An Evolutional Mixture-of-Experts Training Framework via Dense-To-Sparse Gate

Xiaonan Nie, Xupeng Miao, Shijie Cao +7

Mixture-of-experts (MoE) is becoming popular due to its success in improving the model quality, especially in Transformers. By routing tokens with a sparse gate to a few experts (i…