Unlocking optimal batch size schedules using continuous-time control and perturbation theory

Stefan Perko

Stochastic Gradient Descent (SGD) and its variants are almost universally used to train neural networks and to fit a variety of other parametric models. An important hyperparameter…