29 citations · 107 across the 11 of their papers we have counts for
11 papers
Efficient Pipeline Planning for Expedited Distributed DNN Training
Ziyue Luo, Xiaodong Yi, Guoping Long +4
To train modern large DNN models, pipeline parallelism has recently emerged, which distributes the model across GPUs and enables different devices to process different microbatches…
FusionStitching: Boosting Memory Intensive Computations for Deep Learning Workloads
Zhen Zheng, Pengzhan Zhao, Guoping Long +6
We show in this work that memory intensive computations can result in severe performance problems due to off-chip memory access and CPU-GPU context switch overheads in a wide range…
Auto-MAP: A DQN Framework for Exploring Distributed Execution Plans for DNN Workloads
Siyu Wang, Yi Rong, Shiqing Fan +6
The last decade has witnessed growth in the computational requirements for training deep neural networks. Current approaches (e.g., data/model parallelism, pipeline parallelism) pa…
DAPPLE: A Pipelined Data Parallel Approach for Training Large Models
Shiqing Fan, Yi Rong, Chen Meng +10
It is a challenging task to train large DNN models on sophisticated GPU platforms with diversified interconnect capabilities. Recently, pipelined training has been proposed as an e…
FusionStitching: Boosting Execution Efficiency of Memory Intensive Computations for DL Workloads
Guoping Long, Jun Yang, Wei Lin
Performance optimization is the art of continuous seeking a harmonious mapping between the application domain and hardware. Recent years have witnessed a surge of deep learning (DL…
Characterizing Deep Learning Training Workloads on Alibaba-PAI
Mengdi Wang, Chen Meng, Guoping Long +4
Modern deep learning models have been exploited in various domains, including computer vision (CV), natural language processing (NLP), search and recommendation. In practical AI cl…