First order optimization methods based on Hessian-driven Nesterov accelerated gradient flow

Abstract

A novel dynamical inertial Newton system, which is called Hessian-driven Nesterov accelerated gradient (H-NAG) flow is proposed. Convergence of the continuous trajectory are established via tailored Lyapunov function, and new first-order accelerated optimization methods are proposed from ODE solvers. It is shown that (semi-)implicit schemes can always achieve linear rate and explicit schemes have the optimal(accelerated) rates for convex and strongly convex objectives. In particular, Nesterov's optimal method is recovered from an explicit scheme for our H-NAG flow. Furthermore, accelerated splitting algorithms for composite optimization problems are also developed.

References in corpus (5)

• A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
• A Lyapunov Analysis of Momentum Methods in Optimization
• Understanding the Acceleration Phenomenon via High-Resolution Differential Equations
• From differential equation solvers to accelerated first-order methods for convex optimization
• Accelerated First-Order Methods: Differential Equations and Lyapunov Functions

Cited by in corpus (4)

• The long time behavior and the rate of convergence of symplectic convex algorithms obtained via splitting discretizations of inertial damping systems
• Accelerated differential inclusion for convex optimization
• A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate
• Inertial Newton Algorithms Avoiding Strict Saddle Points