CROSSBOW: Scaling Deep Learning with Small Batch Sizes on Multi-GPU Servers

Alexandros Koliousis, Pijika Watcharapichat, Matthias Weidlich +3

Deep learning models are trained on servers with many GPUs, and training must scale with the number of GPUs. Systems such as TensorFlow and Caffe2 train models with parallel synchr…