most citedMegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

24 citations · 24 across the 2 of their papers we have counts for

collaborators
Showing cs.AIShow all

1 paper · 1 filter

cs.AI2024

ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development

Borui Wan, Mingji Han, Yiyao Sheng +9

Checkpointing to preserve training states is crucial during the development of Large Foundation Models (LFMs), for training resumption upon various failures or changes in GPU resou…