Online Batch Selection for Faster Training of Neural Networks

Ilya Loshchilov, Frank Hutter

Deep neural networks are commonly trained using stochastic non-convex optimization procedures, which are driven by gradient information estimated on fractions (batches) of the data…