10 citations · 13 across the 3 of their papers we have counts for
4 papers
DisDP: Disaggregating Compute, Network, and Storage for Model-Sharded Data-Parallel Training
Mo Sun, Zihan Yang, Changyue Liao +5
Model-sharded data parallelism (MSDP), e.g., ZeRO, evenly shards the model states across all GPUs, and thus has been widely adopted by LLM pre-training, such as Llama and DeepSeek,…
LoHan: Low-Cost High-Performance Framework to Fine-Tune 100B Model on a Consumer GPU
Changyue Liao, Mo Sun, Zihan Yang +5
Nowadays, AI researchers become more and more interested in fine-tuning a pre-trained LLM, whose size has grown to up to over 100B parameters, for their downstream tasks. One appro…
Helios: An Efficient Out-of-core GNN Training System on Terabyte-scale Graphs with In-memory Performance
Jie Sun, Mo Sun, Zheng Zhang +6
Training graph neural networks (GNNs) on large-scale graph data holds immense promise for numerous real-world applications but remains a great challenge. Several disk-based GNN sys…
Walle: An End-to-End, General-Purpose, and Large-Scale Production System for Device-Cloud Collaborative Machine Learning
Chengfei Lv, Chaoyue Niu, Renjie Gu +17
To break the bottlenecks of mainstream cloud-based machine learning (ML) paradigm, we adopt device-cloud collaborative ML and build the first end-to-end and general-purpose system,…