Memory Efficient and Staleness Free Pipeline Parallel DNN Training Framework with Improved Convergence Speed

Ankita Dutta, Nabendu Chaki, Rajat K. De

High resource requirement for Deep Neural Network (DNN) training across multiple GPUs necessitates development of various parallelism techniques. In this paper, we introduce two in…