Momentum Improves Optimization on Riemannian Manifolds
arXiv:2002.04144
Abstract
We develop a new Riemannian descent algorithm that relies on momentum to improve over existing first-order methods for geodesically convex optimization. In contrast, accelerated convergence rates proved in prior work have only been shown to hold for geodesically strongly-convex objective functions. We further extend our algorithm to geodesically weakly-quasi-convex objectives. Our proofs of convergence rely on a novel estimate sequence that illustrates the dependency of the convergence rate on the curvature of the manifold. We validate our theoretical results empirically on several optimization problems defined on the sphere and on the manifold of positive definite matrices.
arXiv commentarXiv admin note: text overlap with arXiv:1910.10782
References in corpus (7)
- A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
- A Variational Perspective on Accelerated Methods in Optimization
- Conic geometric optimisation on the manifold of positive definite matrices
- Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds
- Geodesic Convex Optimization: Differentiation on Manifolds, Geodesics, and Convexity
- Towards Riemannian Accelerated Gradient Methods
- From Nesterov's Estimate Sequence to Riemannian Acceleration