A Conservation Law Method in Optimization
arXiv:1708.08035
Abstract
We propose some algorithms to find local minima in nonconvex optimization and to obtain global minima in some degree from the Newton Second Law without friction. With the key observation of the velocity observable and controllable in the motion, the algorithms simulate the Newton Second Law without friction based on symplectic Euler scheme. From the intuitive analysis of analytical solution, we give a theoretical analysis for the high-speed convergence in the algorithm proposed. Finally, we propose the experiments for strongly convex function, non-strongly convex function and nonconvex function in high-dimension.
References in corpus (5)
- A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
- A Variational Perspective on Accelerated Methods in Optimization
- Gradient Descent Learns Linear Dynamical Systems
- A Lyapunov Analysis of Momentum Methods in Optimization
- Efficient approaches for escaping higher order saddle points in non-convex optimization