Gradient descent with generalized Newton's method

Zhiqi Bu, Shiyun Xu

We propose the generalized Newton's method (GeN) -- a Hessian-informed approach that applies to any optimizer such as SGD and Adam, and covers the Newton-Raphson method as a sub-ca…