Study on the Large Batch Size Training of Neural Networks Based on the Second Order Gradient

Fengli Gao, Huicai Zhong

Large batch size training in deep neural networks (DNNs) possesses a well-known 'generalization gap' that remarkably induces generalization performance degradation. However, it rem…