A New Perspective for Understanding Generalization Gap of Deep Neural Networks Trained with Large Batch Sizes

Oyebade K. Oyedotun, Konstantinos Papadopoulos, Djamila Aouada

Deep neural networks (DNNs) are typically optimized using various forms of mini-batch gradient descent algorithm. A major motivation for mini-batch gradient descent is that with a…