52 citations · 171 across the 27 of their papers we have counts for
33 papers
The global convergence time of stochastic gradient descent in non-convex landscapes: Sharp estimates via large deviations
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
In this paper, we examine the time it takes for stochastic gradient descent (SGD) to reach the global minimum of a general, non-convex loss function. We approach this question thro…
: a library for Wasserstein distributionally robust machine learning
Florian Vincent, Waïss Azizian, Franck Iutzeler +1
We present skwdro, a Python library for training robust machine learning models. The library is based on distributionally robust optimization using Wasserstein distances, popular i…
What is the long-run distribution of stochastic gradient descent? A large deviations analysis
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
In this paper, we examine the long-run distribution of stochastic gradient descent (SGD) in general, non-convex problems. Specifically, we seek to understand which regions of the p…
Derivatives of Stochastic Gradient Descent in parametric optimization
Franck Iutzeler, Edouard Pauwels, Samuel Vaiter
We consider stochastic optimization problems where the objective depends on some parameter, as commonly found in hyperparameter optimization for instance. We investigate the behavi…
Delay-tolerant distributed Bregman proximal algorithms
S. Chraibi, F. Iutzeler, J. Malick +1
Many problems in machine learning write as the minimization of a sum of individual loss functions over the training examples. These functions are usually differentiable but, in som…
Exact Generalization Guarantees for (Regularized) Wasserstein Distributionally Robust Models
Waïss Azizian, Franck Iutzeler, Jérôme Malick
Wasserstein distributionally robust estimators have emerged as powerful models for prediction and decision-making under uncertainty. These estimators provide attractive generalizat…
The rate of convergence of Bregman proximal methods: Local geometry vs. regularity vs. sharpness
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
We examine the last-iterate convergence rate of Bregman proximal methods - from mirror descent to mirror-prox and its optimistic variants - as a function of the local geometry indu…
Entropy-regularized Wasserstein distributionally robust shape and topology optimization
Charles Dapogny, Franck Iutzeler, Andrea Meda +1
This brief note aims to introduce the recent paradigm of distributional robustness in the field of shape and topology optimization. Acknowledging that the probability law of uncert…
Harnessing structure in composite nonsmooth minimization
Gilles Bareilles, Franck Iutzeler, Jérôme Malick
We consider the problem of minimizing the composition of a nonsmooth function with a smooth mapping in the case where the proximity operator of the nonsmooth function can be explic…
Push--Pull with Device Sampling
Yu-Guan Hsieh, Yassine Laguel, Franck Iutzeler +1
We consider decentralized optimization problems in which a number of agents collaborate to minimize the average of their local functions by exchanging over an underlying communicat…
Regularization for Wasserstein Distributionally Robust Optimization
Waïss Azizian, Franck Iutzeler, Jérôme Malick
Optimal transport has recently proved to be a useful tool in various machine learning applications needing comparisons of probability measures. Among these, applications of distrib…
Learning over No-Preferred and Preferred Sequence of Items for Robust Recommendation (Extended Abstract)
Aleksandra Burashnikova, Yury Maximov, Marianne Clausel +3
This paper is an extended version of [Burashnikova et al., 2021, arXiv: 2012.06910], where we proposed a theoretically supported sequential strategy for training a large-scale Reco…
The Last-Iterate Convergence Rate of Optimistic Mirror Descent in Stochastic Variational Inequalities
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
In this paper, we analyze the local convergence rate of optimistic mirror descent methods in stochastic variational inequalities, a class of optimization problems with important ap…
Optimization in Open Networks via Dual Averaging
Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick +1
In networks of autonomous agents (e.g., fleets of vehicles, scattered sensors), the problem of minimizing the sum of the agents' local functions has received a lot of interest. We…
Newton acceleration on manifolds identified by proximal-gradient methods
Gilles Bareilles, Franck Iutzeler, Jérôme Malick
Proximal methods are known to identify the underlying substructure of nonsmooth optimization problems. Even more, in many interesting situations, the output of a proximity operator…
Multi-Agent Online Optimization with Delays: Asynchronicity, Adaptivity, and Optimism
Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick +1
In this paper, we provide a general framework for studying multi-agent online learning problems in the presence of delays and asynchronicities. Specifically, we propose and analyze…
Nonsmoothness in Machine Learning: specific structure, proximal identification, and applications
Franck Iutzeler, Jérôme Malick
Nonsmoothness is often a curse for optimization; but it is sometimes a blessing, in particular for applications in machine learning. In this paper, we present the specific structur…
Randomized Progressive Hedging methods for Multi-stage Stochastic Programming
Gilles Bareilles, Yassine Laguel, Dmitry Grishchenko +2
Progressive Hedging is a popular decomposition algorithm for solving multi-stage stochastic optimization problems. A computational bottleneck of this algorithm is that all scenario…
Rank-one partitioning: formalization, illustrative examples, and a new cluster enhancing strategy
Charlotte Laclau, Franck Iutzeler, Ievgen Redko
In this paper, we introduce and formalize a rank-one partitioning learning paradigm that unifies partitioning methods that proceed by summarizing a data set using a single vector t…
Proximal Gradient methods with Adaptive Subspace Sampling
Dmitry Grishchenko, Franck Iutzeler, Jérôme Malick
Many applications in machine learning or signal processing involve nonsmooth optimization problems. This nonsmoothness brings a low-dimensional structure to the optimal solutions.…
Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling
Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick +1
Owing to their stability and convergence speed, extragradient methods have become a staple for solving large-scale saddle-point problems in machine learning. The basic premise of t…
On the Interplay between Acceleration and Identification for the Proximal Gradient algorithm
Gilles Bareilles, Franck Iutzeler
In this paper, we study the interplay between acceleration and structure identification for the proximal gradient algorithm. We report and analyze several cases where this interpla…
On the convergence of single-call stochastic extra-gradient methods
Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick +1
Variational inequalities have recently attracted considerable interest in machine learning as a flexible paradigm for models that go beyond ordinary loss function minimization (suc…
Distributed Learning with Sparse Communications by Identification
Dmitry Grishchenko, Franck Iutzeler, Jérôme Malick +1
In distributed optimization for large-scale learning, a major performance limitation comes from the communications between the different entities. When computations are performed b…