Horovod: fast and easy distributed deep learning in TensorFlow

Alexander Sergeev, Mike Del Balso

Training modern deep learning models requires large amounts of computation, often provided by GPUs. Scaling computation from one GPU to many can enable much faster training and res…