Heterogeneous CPU+GPU Stochastic Gradient Descent Algorithms

Yujing Ma, Florin Rusu

The widely-adopted practice is to train deep learning models with specialized hardware accelerators, e.g., GPUs or TPUs, due to their superior performance on linear algebra operati…