Theano-MPI: a Theano-based Distributed Training Framework

He Ma, Fei Mao, Graham W. Taylor

We develop a scalable and extendable training framework that can utilize GPUs across nodes in a cluster and accelerate the training of deep learning models based on data parallelis…