XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training

Lei Guan, Wotao Yin, Dongsheng Li +1

We propose XPipe, an efficient asynchronous pipeline model parallelism approach for multi-GPU DNN training. XPipe is designed to use multiple GPUs to concurrently and continuously…