29 citations · 75 across the 13 of their papers we have counts for
Showing 2020Show all
3 papers · 1 filter
cs.DC2020★ 12 cited
FusionStitching: Boosting Memory Intensive Computations for Deep Learning Workloads
Zhen Zheng, Pengzhan Zhao, Guoping Long +6
We show in this work that memory intensive computations can result in severe performance problems due to off-chip memory access and CPU-GPU context switch overheads in a wide range…
cs.DC2020★ 5 cited
Auto-MAP: A DQN Framework for Exploring Distributed Execution Plans for DNN Workloads
Siyu Wang, Yi Rong, Shiqing Fan +6
The last decade has witnessed growth in the computational requirements for training deep neural networks. Current approaches (e.g., data/model parallelism, pipeline parallelism) pa…
cs.DC2020★ 29 cited
DAPPLE: A Pipelined Data Parallel Approach for Training Large Models
Shiqing Fan, Yi Rong, Chen Meng +10
It is a challenging task to train large DNN models on sophisticated GPU platforms with diversified interconnect capabilities. Recently, pipelined training has been proposed as an e…