most citedHAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis

16 citations · 22 across the 2 of their papers we have counts for

collaborators
Showing cs.DCShow all

2 papers · 1 filter

cs.DC2024★ 16 cited

HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis

Shiwei Zhang, Lansong Diao, Chuan Wu +3

Single-Program-Multiple-Data (SPMD) parallelism has recently been adopted to train large deep neural networks (DNNs). Few studies have explored its applicability on heterogeneous c…

cs.DC2022★ 6 cited

Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion

Xiaodong Yi, Shiwei Zhang, Lansong Diao +6

This paper proposes DisCo, an automatic deep learning compilation module for data-parallel distributed training. Unlike most deep learning compilers that focus on training or infer…