Revisiting Small Batch Training for Deep Neural Networks

Dominic Masters, Carlo Luschi

Modern deep neural network training is typically based on mini-batch stochastic gradient optimization. While the use of large mini-batches increases the available computational par…