Inertial Newton Algorithms Avoiding Strict Saddle Points
arXiv:2111.04596 · doi:10.1007/s10957-023-02330-0
Abstract
We study the asymptotic behavior of second-order algorithms mixing Newton's method and inertial gradient descent in non-convex landscapes. We show that, despite the Newtonian behavior of these methods, they almost always escape strict saddle points. We also evidence the role played by the hyper-parameters of these methods in their qualitative behavior near critical points. The theoretical results are supported by numerical illustrations.
References in corpus (4)
- The NumPy array: a structure for efficient numerical computation
- A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
- Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
- First order optimization methods based on Hessian-driven Nesterov accelerated gradient flow