185 citations · 295 across the 17 of their papers we have counts for
17 papers
Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation
Robert M. Gower, Guillaume Garrigos, Nicolas Loizou +3
We provide a general convergence theorem of an idealized stochastic Polyak step size called SPS. Besides convexity, we only assume a local expected gradient bound, that include…
Tracking the Median of Gradients with a Stochastic Proximal Point Method
Fabian Schaipp, Guillaume Garrigos, Umut Simsekli +1
There are several applications of stochastic optimization where one can benefit from a robust estimate of the gradient. For example, domains such as distributed learning with corru…
Function Value Learning: Adaptive Learning Rates Based on the Polyak Stepsize and Function Splitting in ERM
Guillaume Garrigos, Robert M. Gower, Fabian Schaipp
Here we develop variants of SGD (stochastic gradient descent) with an adaptive step size that make use of the sampled loss values. In particular, we focus on solving a finite sum-o…
Online Inventory Problems: Beyond the i.i.d. Setting with Online Convex Optimization
Massil Hihat, Stéphane Gaïffas, Guillaume Garrigos +1
We study multi-product inventory control problems where a manager makes sequential replenishment decisions based on partial historical information in order to minimize its cumulati…
Provable convergence guarantees for black-box variational inference
Justin Domke, Guillaume Garrigos, Robert Gower
Black-box variational inference is widely used in situations where there is no proof that its stochastic optimization succeeds. We suggest this is due to a theoretical gap in exist…
Square distance functions are Polyak-Łojasiewicz and vice-versa
Guillaume Garrigos
This short note gathers known results to state that the squared distance function to a (nonconvex) closed set of an Euclidean space is Polyak-Łojasiewicz. As a fuzzy reciprocate, w…
Handbook of Convergence Theorems for (Stochastic) Gradient Methods
Guillaume Garrigos, Robert M. Gower
This is a handbook of simple proofs of the convergence of gradient and stochastic gradient descent type methods. We consider functions that are Lipschitz, smooth, convex, strongly…
SAN: Stochastic Average Newton Algorithm for Minimizing Finite Sums
Jiabin Chen, Rui Yuan, Guillaume Garrigos +1
We present a principled approach for designing stochastic Newton methods for solving finite sum optimization problems. Our approach has two steps. First, we re-write the stationari…
Accelerated iterative regularization via dual diagonal descent
Luca Calatroni, Guillaume Garrigos, Lorenzo Rosasco +1
We propose and analyze an accelerated iterative dual diagonal descent algorithm for the solution of linear inverse problems with general regularization and data-fit functions. In p…
Sparse Multiple Kernel Learning: Support Identification via Mirror Stratifiability
Guillaume Garrigos, Lorenzo Rosasco, Silvia Villa
In statistical machine learning, kernel methods allow to consider infinite dimensional feature spaces with a computational cost that only depends on the number of observations. Thi…
Model Consistency for Learning with Mirror-Stratifiable Regularizers
Jalal Fadili, Guillaume Garrigos, Jérome Malick +1
Low-complexity non-smooth convex regularizers are routinely used to impose some structure (such as sparsity or low-rank) on the coefficients for linear predictors in supervised lea…
Thresholding gradient methods in Hilbert spaces: support identification and linear convergence
Guillaume Garrigos, Lorenzo Rosasco, Silvia Villa
We study regularized least squares optimization problem in a separable Hilbert space. We show that the iterative soft-thresholding algorithm (ISTA) converges linearly, wit…
Convergence of the Forward-Backward Algorithm: Beyond the Worst Case with the Help of Geometry
Guillaume Garrigos, Lorenzo Rosasco, Silvia Villa
We provide a comprehensive study of the convergence of the forward-backward algorithm under suitable geometric conditions, such as conditioning or Łojasiewicz properties. These geo…
Iterative regularization via dual diagonal descent
Guillaume Garrigos, Lorenzo Rosasco, Silvia Villa
In the context of linear inverse problems, we propose and study a general iterative regularization method allowing to consider large classes of regularizers and data-fit terms. The…
Multiibjective optimization : an inertial dynamical approach to Pareto optima
Hédy Attouch, Guillaume Garrigos
We present some first results concerning a gradient-based dynamic approach to multi-objective optimization problems, involving inertial effects. We prove the existence of global so…
A dynamic gradient approach to Pareto optimization with nonsmooth convex objective functions
Hedy Attouch, Guillaume Garrigos, Xavier Goudou
In a general Hilbert framework, we consider continuous gradient-like dynamical systems for constrained multiobjective optimization involving non-smooth convex objective functions.…
Splitting methods with variable metric for KL functions
Pierre Frankel, Guillaume Garrigos, Juan Peypouquet
We study the convergence of general abstract descent methods applied to a lower semicontinuous nonconvex function f that satisfies the Kurdyka-Lojasiewicz inequality in a Hilbert s…