FlashRecovery: Fast and Low-Cost Recovery from Failures for Large-Scale Training of LLMs

Haijun Zhang, Jinxiang Wang, Zhenhua Yu +20

Large language models (LLMs) have made a profound impact across various fields due to their advanced capabilities. However, training these models at unprecedented scales requires e…