Adaptive Hamiltonian Variational Integrators and Symplectic Accelerated Optimization
arXiv:1709.01975
Abstract
It is well known that symplectic integrators lose their near energy preservation properties when variable step sizes are used. The most common approach to combine adaptive step sizes and symplectic integrators involves the Poincaré transformation of the original Hamiltonian. In this article, we provide a framework for the construction of variational integrators using the Poincaré transformation. Since the transformed Hamiltonian is typically degenerate, the use of Hamiltonian variational integrators based on Type II or Type III generating functions is required instead of the more traditional Lagrangian variational integrators based on Type I generating functions. Error analysis is provided and numerical tests based on the Taylor variational integrator approach of Schmitt, Shingel, Leok (2018) to time-adaptive variational integration of Kepler's 2-Body problem are presented. Finally, we use our adaptive framework together with the variational approach to accelerated optimization presented in Wibisono, Wilson, Jordan (2016) to design efficient variational and non-variational explicit integrators for symplectic accelerated optimization.
arXiv comment32 pages, 8 figures. To appear in SIAM Journal on Scientific Computing
References in corpus (8)
- A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
- Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
- A Variational Perspective on Accelerated Methods in Optimization
- Stochastic Variational Integrators
- Explicit symplectic approximation of nonseparable Hamiltonians: algorithm and long time performance
- Explicit methods in extended phase space for inseparable Hamiltonian problems
- Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent
- On Symplectic Optimization