Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems

Tianle Cai, Ruiqi Gao, Jikai Hou +5

First-order methods such as stochastic gradient descent (SGD) are currently the standard algorithm for training deep neural networks. Second-order methods, despite their better con…