Optimization on manifolds: A symplectic approach

Abstract

Optimization tasks are crucial in statistical machine learning. Recently, there has been great interest in leveraging tools from dynamical systems to derive accelerated and robust optimization methods via suitable discretizations of continuous-time systems. However, these ideas have mostly been limited to Euclidean spaces and unconstrained settings, or to Riemannian gradient flows. In this work, we propose a dissipative extension of Dirac's theory of constrained Hamiltonian systems as a general framework for solving optimization problems over smooth manifolds, including problems with nonlinear constraints. We develop geometric/symplectic numerical integrators on manifolds that are "rate-matching," i.e., preserve the continuous-time rates of convergence. In particular, we introduce a dissipative RATTLE integrator able to achieve optimal convergence rate locally. Our class of (accelerated) algorithms are not only simple and efficient but also applicable to a broad range of contexts.

additional results, including rates for constrained optimization on manifolds

References in corpus (4)

• Gradient flows and proximal splitting methods: A unified view on accelerated and stochastic optimization
• Hamiltonian Monte Carlo on Symmetric and Homogeneous Spaces via Symplectic Reduction
• Manifold lifting: scaling MCMC to the vanishing noise regime
• On Constraints in First-Order Optimization: A View from Non-Smooth Dynamical Systems

Cited by in corpus (1)

• Geometric Methods for Sampling, Optimisation, Inference and Adaptive Agents