59 citations · 195 across the 37 of their papers we have counts for
25 papers · 1 filter
Variance reduction techniques for stochastic proximal point algorithms
Cheik Traoré, Vassilis Apidopoulos, Saverio Salzo +1
In the context of finite sums minimization, variance reduction techniques are widely used to improve the performance of state-of-the-art stochastic gradient methods. Their practica…
Regularization properties of dual subgradient flow
Vassilis Apidopoulos, Cesare Molinari, Lorenzo Rosasco +1
Dual gradient descent combined with early stopping represents an efficient alternative to the Tikhonov variational approach when the regularizer is strongly convex. However, for ma…
An Optimal Structured Zeroth-order Algorithm for Non-smooth Optimization
Marco Rando, Cesare Molinari, Lorenzo Rosasco +1
Finite-difference methods are a class of algorithms designed to solve black-box optimization problems by approximating a gradient of the target function on a set of directions. In…
On the convergence of proximal gradient methods for convex simple bilevel optimization
Puya Latafat, Andreas Themelis, Silvia Villa +1
This paper studies proximal gradient iterations for solving simple bilevel optimization problems where both the upper and the lower level cost functions are split as the sum of dif…
Stochastic Zeroth order Descent with Structured Directions
Marco Rando, Cesare Molinari, Silvia Villa +1
We introduce and analyze Structured Stochastic Zeroth order Descent (S-SZD), a finite difference approach that approximates a stochastic gradient on a set of orthogonal d…
Fast iterative regularization by reusing data
Cristian Vega, Cesare Molinari, Lorenzo Rosasco +1
Discrete inverse problems correspond to solving a system of equations in a stable way with respect to noise in the data. A typical approach to enforce uniqueness and select a meani…
Iterative regularization for low complexity regularizers
Cesare Molinari, Mathurin Massias, Lorenzo Rosasco +1
Iterative regularization exploits the implicit bias of an optimization algorithm to regularize ill-posed problems. Constructing algorithms with such built-in regularization mechani…
Convergence of an Asynchronous Block-Coordinate Forward-Backward Algorithm for Convex Composite Optimization
Cheik Traoré, Saverio Salzo, Silvia Villa
In this paper, we study the convergence properties of a randomized block-coordinate descent algorithm for the minimization of a composite convex objective function, where the block…
Convergence rates for the Heavy-Ball continuous dynamics for non-convex optimization, under Polyak-Łojasiewicz condition
Vassilis Apidopoulos, Nicolò Ginatta, Silvia Villa
We study convergence of the trajectories of the Heavy Ball dynamical system, with constant damping coefficient, in the framework of convex and non-convex smooth optimization. By us…
Zeroth order optimization with orthogonal random directions
David Kozak, Cesare Molinari, Lorenzo Rosasco +2
We propose and analyze a randomized zeroth-order approach based on approximating the exact gradient byfinite differences computed in a set of orthogonal random directions that chan…
Accelerated iterative regularization via dual diagonal descent
Luca Calatroni, Guillaume Garrigos, Lorenzo Rosasco +1
We propose and analyze an accelerated iterative dual diagonal descent algorithm for the solution of linear inverse problems with general regularization and data-fit functions. In p…
Parallel Random Block-Coordinate Forward-Backward Algorithm: A Unified Convergence Analysis
Saverio Salzo, Silvia Villa
We study the block-coordinate forward-backward algorithm in which the blocks are updated in a random and possibly parallel manner, according to arbitrary probabilities. The algorit…
Sparse Multiple Kernel Learning: Support Identification via Mirror Stratifiability
Guillaume Garrigos, Lorenzo Rosasco, Silvia Villa
In statistical machine learning, kernel methods allow to consider infinite dimensional feature spaces with a computational cost that only depends on the number of observations. Thi…
Thresholding gradient methods in Hilbert spaces: support identification and linear convergence
Guillaume Garrigos, Lorenzo Rosasco, Silvia Villa
We study regularized least squares optimization problem in a separable Hilbert space. We show that the iterative soft-thresholding algorithm (ISTA) converges linearly, wit…
Don't relax: early stopping for convex regularization
Simon Matet, Lorenzo Rosasco, Silvia Villa +1
We consider the problem of designing efficient regularization algorithms when regularization is encoded by a (strongly) convex functional. Unlike classical penalization methods bas…
Convergence of the Forward-Backward Algorithm: Beyond the Worst Case with the Help of Geometry
Guillaume Garrigos, Lorenzo Rosasco, Silvia Villa
We provide a comprehensive study of the convergence of the forward-backward algorithm under suitable geometric conditions, such as conditioning or Łojasiewicz properties. These geo…
Iterative regularization via dual diagonal descent
Guillaume Garrigos, Lorenzo Rosasco, Silvia Villa
In the context of linear inverse problems, we propose and study a general iterative regularization method allowing to consider large classes of regularizers and data-fit terms. The…
A first-order stochastic primal-dual algorithm with correction step
Lorenzo Rosasco, Silvia Villa, Bang Cong Vu
We investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive operator and…
Modified Fejér sequences and applications
Junhong Lin, Lorenzo Rosasco, Silvia Villa +1
In this note, we propose and study the notion of modified Fejér sequences. Within a Hilbert space setting, we show that it provides a unifying framework to prove convergence rates…
Stochastic inertial primal-dual algorithms
Lorenzo Rosasco, Silvia Villa, Bang Cong Vu
We propose and study a novel stochastic inertial primal-dual approach to solve composite optimization problems. These latter problems arise naturally when learning with penalized r…
A stochastic inertial forward-backward splitting algorithm for multivariate monotone inclusions
Lorenzo Rosasco, Silvia Villa, Bang Cong Vu
We propose an inertial forward-backward splitting algorithm to compute the zero of a sum of two monotone operators allowing for stochastic errors in the computation of the operator…
A Stochastic forward-backward splitting method for solving monotone inclusions in Hilbert spaces
Lorenzo Rosasco, Silvia Villa, Bang Công Vũ
We propose and analyze the convergence of a novel stochastic forward-backward splitting algorithm for solving monotone inclusions given by the sum of a maximal monotone operator an…
Convergence of Stochastic Proximal Gradient Algorithm
Lorenzo Rosasco, Silvia Villa, Bang Công Vũ
We prove novel convergence results for a stochastic proximal gradient algorithm suitable for solving a large class of convex optimization problems, where a convex objective functio…
Proximal methods for the latent group lasso penalty
Silvia Villa, Lorenzo Rosasco, Sofia Mosci +1
We consider a regularized least squares problem, with regularization by structured sparsity-inducing norms, which extend the usual and the group lasso penalty, by allowing…