A closed loop gradient descent algorithm applied to Rosenbrock's function

Abstract

We introduce a novel adaptive damping technique for an inertial gradient system which finds application as a gradient descent algorithm for unconstrained optimisation. In an example using the non-convex Rosenbrock's function, we show an improvement on existing momentum-based gradient optimisation methods. Also using Lyapunov stability analysis, we demonstrate the performance of the continuous-time version of the algorithm. Using numerical simulations, we consider the performance of its discrete-time counterpart obtained by using the symplectic Euler method of discretisation.

arXiv commentIn proceedings of the 2021 Australia and New Zealand Control Conference, 2021. (Final reviewed version)

References in corpus (5)

• A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
• A Variational Perspective on Accelerated Methods in Optimization
• A Dynamical Systems Perspective on Nesterov Acceleration
• A general system of differential equations to model first order adaptive algorithms
• Continuous Time Analysis of Momentum Methods