StageFrontier: Synchronization-Aware Stage Accounting for Distributed ML Training

Boram Yoon, Wei Chen, Ville Kallioniemi

When a distributed training job slows down, the hard part is knowing where to look. Synchronization hides the cause: a stall on one rank shows up as a wait on the others, so a data…