Accelerating Deep Neural Network Training with Inconsistent Stochastic Gradient Descent

Linnan Wang, Yi Yang, Martin Renqiang Min +1

SGD is the widely adopted method to train CNN. Conceptually it approximates the population with a randomly sampled batch; then it evenly trains batches by conducting a gradient upd…