Improving the convergence of SGD through adaptive batch sizes

Scott Sievert, Shrey Shah

Mini-batch stochastic gradient descent (SGD) and variants thereof approximate the objective function's gradient with a small number of training examples, aka the batch size. Small…