76 citations · 93 across the 15 of their papers we have counts for
15 papers
Finding Simple Proofs for First-Order Optimization
Daniel Berg Thomsen, Manu Upadhyaya, Baptiste Goujaud +2
Progress in mathematics often requires more than a certificate of truth: it requires proof structures that are transparent, checkable, and reusable. Automated systems can increasin…
Gradient Descent on Logistic Regression: Do Large Step-Sizes Work with Data on the Sphere?
Si Yi Meng, Baptiste Goujaud, Antonio Orvieto +1
Gradient descent on logistic regression has many fascinating properties not fully captured by classical analyses. For linearly separable data, the iterates converge in direction to…
Open Problem: Two Riddles in Heavy-Ball Dynamics
Baptiste Goujaud, Adrien Taylor, Aymeric Dieuleveut
This short paper presents two open problems on the widely used Polyak's Heavy-Ball algorithm. The first problem is the method's ability to exactly \textit{accelerate} in dimension…
On Fundamental Proof Structures in First-Order Optimization
Baptiste Goujaud, Aymeric Dieuleveut, Adrien Taylor
First-order optimization methods have attracted a lot of attention due to their practical success in many applications, including in machine learning. Obtaining convergence guarant…
Proving Linear Mode Connectivity of Neural Networks via Optimal Transport
Damien Ferbach, Baptiste Goujaud, Gauthier Gidel +1
The energy landscape of high-dimensional non-convex optimization problems is crucial to understanding the effectiveness of modern deep neural network architectures. Recent works ha…
Provable non-accelerations of the heavy-ball method
Baptiste Goujaud, Adrien Taylor, Aymeric Dieuleveut
In this work, we show that the heavy-ball ($\HB$) method provably does not reach an accelerated convergence rate on smooth strongly convex problems. More specifically, we show that…
Counter-examples in first-order optimization: a constructive approach
Baptiste Goujaud, Aymeric Dieuleveut, Adrien Taylor
While many approaches were developed for obtaining worst-case complexity bounds for first-order optimization methods in the last years, there remain theoretical gaps in cases where…
Quadratic minimization: from conjugate gradient to an adaptive Heavy-ball method with Polyak step-sizes
Baptiste Goujaud, Adrien Taylor, Aymeric Dieuleveut
In this work, we propose an adaptive variation on the classical Heavy-ball method for convex quadratic minimization. The adaptivity crucially relies on so-called "Polyak step-sizes…
Optimal first-order methods for convex functions with a quadratic upper bound
Baptiste Goujaud, Adrien Taylor, Aymeric Dieuleveut
We analyze worst-case convergence guarantees of first-order optimization methods over a function class extending that of smooth and convex functions. This class contains convex fun…
Gradient Descent Is Optimal Under Lower Restricted Secant Inequality And Upper Error Bound
Charles Guille-Escuret, Adam Ibrahim, Baptiste Goujaud +1
The study of first-order optimization is sensitive to the assumptions made on the objective functions. These assumptions induce complexity classes which play a key role in worst-ca…
PEPit: computer-assisted worst-case analyses of first-order optimization methods in Python
Baptiste Goujaud, Céline Moucer, François Glineur +3
PEPit is a Python package aiming at simplifying the access to worst-case analyses of a large family of first-order optimization methods possibly involving gradient, projection, pro…
Super-Acceleration with Cyclical Step-sizes
Baptiste Goujaud, Damien Scieur, Aymeric Dieuleveut +2
We develop a convergence-rate analysis of momentum with cyclical step-sizes. We show that under some assumption on the spectral gap of Hessians in machine learning, cyclical step-s…
A Study of Condition Numbers for First-Order Optimization
Charles Guille-Escuret, Baptiste Goujaud, Manuela Girotti +1
The study of first-order optimization algorithms (FOA) typically starts with assumptions on the objective functions, most commonly smoothness and strong convexity. These metrics ar…
Gradient based sample selection for online continual learning
Rahaf Aljundi, Min Lin, Baptiste Goujaud +1
A continual learning agent learns online with a non-stationary and never-ending stream of data. The key to such learning process is to overcome the catastrophic forgetting of previ…
Robust Detection of Covariate-Treatment Interactions in Clinical Trials
Baptiste Goujaud, Eric W. Tramel, Pierre Courtiol +2
Detection of interactions between treatment effects and patient descriptors in clinical trials is critical for optimizing the drug development process. The increasing volume of dat…