25 citations · 57 across the 14 of their papers we have counts for
14 papers
How Does the Pretraining Distribution Shape In-Context Learning? A Fundamental Trade-Off
Waïss Azizian, Ali Hasan
The factors driving the performance of in-context learning (ICL) in large language models (LLMs) remain poorly understood despite ICL's surprising effectiveness, enabling models to…
The Geometries of Truth Are Orthogonal Across Tasks
Waiss Azizian, Michael Kirchhof, Eugene Ndiaye +4
Large Language Models (LLMs) have demonstrated impressive generalization capabilities across various tasks, but their claim to practical relevance is still mired by concerns on the…
The global convergence time of stochastic gradient descent in non-convex landscapes: Sharp estimates via large deviations
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
In this paper, we examine the time it takes for stochastic gradient descent (SGD) to reach the global minimum of a general, non-convex loss function. We approach this question thro…
: a library for Wasserstein distributionally robust machine learning
Florian Vincent, Waïss Azizian, Franck Iutzeler +1
We present skwdro, a Python library for training robust machine learning models. The library is based on distributionally robust optimization using Wasserstein distances, popular i…
What is the long-run distribution of stochastic gradient descent? A large deviations analysis
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
In this paper, we examine the long-run distribution of stochastic gradient descent (SGD) in general, non-convex problems. Specifically, we seek to understand which regions of the p…
Almost sure convergence rates of stochastic gradient methods under gradient domination
Simon Weissmann, Sara Klein, Waïss Azizian +1
Stochastic gradient methods are among the most important algorithms in training machine learning problems. While classical assumptions such as strong convexity allow a simple analy…
Automatic Rao-Blackwellization for Sequential Monte Carlo with Belief Propagation
Waïss Azizian, Guillaume Baudart, Marc Lelarge
Exact Bayesian inference on state-space models~(SSM) is in general untractable, and unfortunately, basic Sequential Monte Carlo~(SMC) methods do not yield correct approximations fo…
Exact Generalization Guarantees for (Regularized) Wasserstein Distributionally Robust Models
Waïss Azizian, Franck Iutzeler, Jérôme Malick
Wasserstein distributionally robust estimators have emerged as powerful models for prediction and decision-making under uncertainty. These estimators provide attractive generalizat…
The rate of convergence of Bregman proximal methods: Local geometry vs. regularity vs. sharpness
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
We examine the last-iterate convergence rate of Bregman proximal methods - from mirror descent to mirror-prox and its optimistic variants - as a function of the local geometry indu…
Regularization for Wasserstein Distributionally Robust Optimization
Waïss Azizian, Franck Iutzeler, Jérôme Malick
Optimal transport has recently proved to be a useful tool in various machine learning applications needing comparisons of probability measures. Among these, applications of distrib…
The Last-Iterate Convergence Rate of Optimistic Mirror Descent in Stochastic Variational Inequalities
Waïss Azizian, Franck Iutzeler, Jérôme Malick +1
In this paper, we analyze the local convergence rate of optimistic mirror descent methods in stochastic variational inequalities, a class of optimization problems with important ap…
Accelerating Smooth Games by Manipulating Spectral Shapes
Waïss Azizian, Damien Scieur, Ioannis Mitliagkas +2
We use matrix iteration theory to characterize acceleration in smooth games. We define the spectral shape of a family of games as the set containing all eigenvalues of the Jacobian…
Linear Lower Bounds and Conditioning of Differentiable Games
Adam Ibrahim, Waïss Azizian, Gauthier Gidel +1
Recent successes of game-theoretic formulations in ML have caused a resurgence of research interest in differentiable games. Overwhelmingly, that research focuses on methods and up…
A Tight and Unified Analysis of Gradient-Based Methods for a Whole Spectrum of Games
Waïss Azizian, Ioannis Mitliagkas, Simon Lacoste-Julien +1
We consider differentiable games where the goal is to find a Nash equilibrium. The machine learning community has recently started using variants of the gradient method (GD). Prime…