34 citations · 102 across the 12 of their papers we have counts for
4 papers · 1 filter
Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion
Xiaodong Yi, Shiwei Zhang, Lansong Diao +6
This paper proposes DisCo, an automatic deep learning compilation module for data-parallel distributed training. Unlike most deep learning compilers that focus on training or infer…
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training
Hanpeng Hu, Chenyu Jiang, Yuchen Zhong +5
Distributed training using multiple devices (e.g., GPUs) has been widely adopted for learning DNN models over large datasets. However, the performance of large-scale distributed tr…
Optimizing Task Placement and Online Scheduling for Distributed GNN Training Acceleration
Ziyue Luo, Yixin Bao, Chuan Wu
Training Graph Neural Networks (GNN) on large graphs is resource-intensive and time-consuming, mainly due to the large graph data that cannot be fit into the memory of a single mac…
Efficient Pipeline Planning for Expedited Distributed DNN Training
Ziyue Luo, Xiaodong Yi, Guoping Long +4
To train modern large DNN models, pipeline parallelism has recently emerged, which distributes the model across GPUs and enables different devices to process different microbatches…