SGD with Partial Hessian for Deep Neural Networks Optimization

Ying Sun, Hongwei Yong, Lei Zhang

Due to the effectiveness of second-order algorithms in solving classical optimization problems, designing second-order optimizers to train deep neural networks (DNNs) has attracted…