Accurate Simulation of Distributed Training Jobs with Network Contention Modeling

Yeonho Yoo, Hyunho Lee, Hyunmok Choi +2

Trace-driven simulation is widely used to evaluate distributed training (DT) jobs in GPU clusters, but existing simulators either ignore network contention or approximate it with a…