29 citations · 39 across the 4 of their papers we have counts for
4 papers
DISC: A Dynamic Shape Compiler for Machine Learning Workloads
Kai Zhu, Wenyi Zhao, Zhen Zheng +8
Many recent machine learning models show dynamic shape characteristics. However, existing AI compiler optimization systems suffer a lot from problems brought by dynamic shape model…
Auto-MAP: A DQN Framework for Exploring Distributed Execution Plans for DNN Workloads
Siyu Wang, Yi Rong, Shiqing Fan +6
The last decade has witnessed growth in the computational requirements for training deep neural networks. Current approaches (e.g., data/model parallelism, pipeline parallelism) pa…
DAPPLE: A Pipelined Data Parallel Approach for Training Large Models
Shiqing Fan, Yi Rong, Chen Meng +10
It is a challenging task to train large DNN models on sophisticated GPU platforms with diversified interconnect capabilities. Recently, pipelined training has been proposed as an e…
DaSGD: Squeezing SGD Parallelization Performance in Distributed Training Using Delayed Averaging
Qinggang Zhou, Yawen Zhang, Pengcheng Li +4
The state-of-the-art deep learning algorithms rely on distributed training systems to tackle the increasing sizes of models and training data sets. Minibatch stochastic gradient de…