CrossoverScheduler: Overlapping Multiple Distributed Training Applications in a Crossover Manner

Cheng Luo, Lei Qu, Youshan Miao +2

Distributed deep learning workloads include throughput-intensive training tasks on the GPU clusters, where the Distributed Stochastic Gradient Descent (SGD) incurs significant comm…