Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum

Keisuke Kamo, Hideaki Iiduka

Stochastic gradient descent with momentum (SGDM), in which a momentum term is added to SGD, has been well studied in both theory and practice. The theoretical studies show that the…