Pipelined Training with Stale Weights of Deep Convolutional Neural Networks

Lifu Zhang, Tarek S. Abdelrahman

The growth in the complexity of Convolutional Neural Networks (CNNs) is increasing interest in partitioning a network across multiple accelerators during training and pipelining th…