most citedFault-Tolerant Hybrid-Parallel Training at Scale with Reliable and Efficient In-memory Checkpointing

1 citations · 1 across the 2 of their papers we have counts for

collaborators
Showing cs.DCShow all

2 papers · 1 filter

cs.DC2025

ElasWave: An Elastic-Native System for Scalable Hybrid-Parallel Training

Xueze Kang, Guangyu Xiang, Yuxin Wang +16

Large-scale LLM pretraining now runs across -- accelerators, making failures routine and elasticity mandatory. We posit that an elastic-native training system must join…

cs.DC2023★ 1 cited

Fault-Tolerant Hybrid-Parallel Training at Scale with Reliable and Efficient In-memory Checkpointing

Yuxin Wang, Xueze Kang, Shaohuai Shi +8

To efficiently scale large model (LM) training, researchers transition from data parallelism (DP) to hybrid parallelism (HP) on GPU clusters, which frequently experience hardware a…