Showing cs.DCShow all
2 papers · 1 filter
cs.DC2025
FlashRecovery: Fast and Low-Cost Recovery from Failures for Large-Scale Training of LLMs
Haijun Zhang, Jinxiang Wang, Zhenhua Yu +20
Large language models (LLMs) have made a profound impact across various fields due to their advanced capabilities. However, training these models at unprecedented scales requires e…
cs.DC2023
DistSim: A performance model of large-scale hybrid distributed DNN training
Guandong Lu, Runzhe Chen, Yakai Wang +8
With the ever-increasing computational demand of DNN training workloads, distributed training has been widely adopted. A combination of data, model and pipeline parallelism strateg…