Doing More by Doing Less: How Structured Partial Backpropagation Improves Deep Learning Clusters

Adarsh Kumar, Kausik Subramanian, Shivaram Venkataraman +1

Many organizations employ compute clusters equipped with accelerators such as GPUs and TPUs for training deep learning models in a distributed fashion. Training is resource-intensi…