A Generic online acceleration scheme for Optimization algorithms via Relaxation and Inertia

Abstract

We propose generic acceleration schemes for a wide class of optimization and iterative schemes based on relaxation and inertia. In particular, we introduce methods that automatically tunes the acceleration coefficients online, and establish their convergence. This is made possible by considering the class of fixed-points iterations over averaged operators which encompass gradient methods, ADMM, primal dual algorithms, an so on.

References in corpus (5)

• The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than $1/k^2$
• From Averaging to Acceleration, There is Only a Step-size
• Asynchronous Distributed Optimization using a Randomized Alternating Direction Method of Multipliers
• Local Linear Convergence of ISTA and FISTA on the LASSO Problem
• Line Search for Averaged Operator Iteration

Cited by in corpus (2)

• A distributed primal-dual algorithm for computation of generalized Nash equilibria with shared affine coupling constraints via operator splitting methods
• An Inertial Parallel and Asynchronous Fixed-Point Iteration for Convex Optimization