TiMePReSt: Time and Memory Efficient Pipeline Parallel DNN Training with Removed Staleness

Ankita Dutta, Nabendu Chaki, Rajat K. De

DNN training is time-consuming and requires efficient multi-accelerator parallelization, where a single training iteration is split over available accelerators. Current approaches…