57 citations · 168 across the 16 of their papers we have counts for
Showing cs.DCShow all
3 papers · 1 filter
cs.DC2023
OSDP: Optimal Sharded Data Parallel for Distributed Deep Learning
Youhe Jiang, Fangcheng Fu, Xupeng Miao +2
Large-scale deep learning models contribute to significant performance improvements on varieties of downstream tasks. Current data and model parallelism approaches utilize model re…
cs.DC2023★ 38 cited
FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement
Xiaonan Nie, Xupeng Miao, Zilong Wang +5
With the increasing data volume, there is a trend of using large-scale pre-trained models to store the knowledge into an enormous number of model parameters. The training of these…
cs.DC2021★ 1 cited
K-Core Decomposition on Super Large Graphs with Limited Resources
Shicheng Gao, Jie Xu, Xiaosen Li +5
K-core decomposition is a commonly used metric to analyze graph structure or study the relative importance of nodes in complex graphs. Recent years have seen rapid growth in the sc…