7 papers · 1 filter
Step-Size Stability in Stochastic Optimization: A Theoretical Perspective
Fabian Schaipp, Robert M. Gower, Adrien Taylor
We present a theoretical analysis of stochastic optimization methods in terms of their sensitivity with respect to the step size. We identify a key quantity that, for each method,…
On the convergence rate of the boosted Difference-of-Convex Algorithm (DCA)
Hadi Abbaszadehpeivasti, Etienne de Klerk, Adrien Taylor
The difference-of-convex algorithm (DCA) is a well-established nonlinear programming technique that solves successive convex optimization problems. These sub-problems are obtained…
The AutoLyap software suite for computer-assisted Lyapunov analyses of first-order methods
Manu Upadhyaya, Shuvomoy Das Gupta, Adrien B. Taylor +2
We introduce AutoLyap, a software suite that assists with Lyapunov analyses of a wide class of first-order methods for structured optimization and inclusion problems. Lyapunov anal…
Optimized projection-free algorithms for online learning: construction and worst-case analysis
Julien Weibel, Pierre Gaillard, Wouter M. Koolen +1
This work studies and develop projection-free algorithms for online learning with linear optimization oracles (a.k.a. Frank-Wolfe) for handling the constraint set. More precisely,…
A constructive approach to strengthen algebraic descriptions of function and operator classes
Anne Rubbens, Julien M. Hendrickx, Adrien Taylor
It is well known that functions (resp. operators) satisfying a property~ on a subset cannot necessarily be extended to a function (resp. operator) satisf…
Geometry-dependent matching pursuit: a transition phase for convergence on linear regression and LASSO
Céline Moucer, Adrien Taylor, Francis Bach
Greedy first-order methods, such as coordinate descent with Gauss-Southwell rule or matching pursuit, have become popular in optimization due to their natural tendency to propose s…
A note on approximate accelerated forward-backward methods with absolute and relative errors, and possibly strongly convex objectives
Mathieu Barré, Adrien Taylor, Francis Bach
In this short note, we provide a simple version of an accelerated forward-backward method (a.k.a. Nesterov's accelerated proximal gradient method) possibly relying on approximate p…