A Continuous-time Perspective for Modeling Acceleration in Riemannian Optimization
arXiv:1910.10782
Abstract
We propose a novel second-order ODE as the continuous-time limit of a Riemannian accelerated gradient-based method on a manifold with curvature bounded from below. This ODE can be seen as a generalization of the ODE derived for Euclidean spaces, and can also serve as an analysis tool. We study the convergence behavior of this ODE for different classes of functions, such as geodesically convex, strongly-convex and weakly-quasi-convex. We demonstrate how such an ODE can be discretized using a semi-implicit and Nesterov-inspired numerical integrator, that empirically yields stable algorithms which are faithful to the continuous-time analysis and exhibit accelerated convergence.
International Conference on Artificial Intelligence and Statistics (AISTATS) 2020
References in corpus (11)
- A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
- A Variational Perspective on Accelerated Methods in Optimization
- Conic geometric optimisation on the manifold of positive definite matrices
- A Lyapunov Analysis of Momentum Methods in Optimization
- Understanding the Acceleration Phenomenon via High-Resolution Differential Equations
- Hamiltonian Descent Methods
- On Symplectic Optimization
- Efficiently escaping saddle points on manifolds
- Towards Riemannian Accelerated Gradient Methods
- Escaping from saddle points on Riemannian manifolds
- Shadowing Properties of Optimization Algorithms