Guard: Scalable Straggler Detection and Node Health Management for Large-Scale Training

Guanliang Liu, Abhinandan Patni, Congzhu Lin +14

Training frontier-scale foundation models involves coordinating tens of thousands of GPUs over multi-month runs, where even minor performance degradations can accumulate into subst…