BaPipe: Exploration of Balanced Pipeline Parallelism for DNN Training

Letian Zhao, Rui Xu, Tianqi Wang +5

The size of deep neural networks (DNNs) grows rapidly as the complexity of the machine learning algorithm increases. To satisfy the requirement of computation and memory of DNN tra…