16 citations · 29 across the 14 of their papers we have counts for
14 papers
Differentiating Minimal-Norm Solutions to Parametric Optimization Problems
Baptiste Plaquevent-Jourdain, Jalal Fadili, Antonio Silveti-Falls
Differentiating through parametric optimization problems is central to bilevel programming and meta-learning, often accomplished using approximate implicit differentiation. The imp…
Frank-Wolfe with Moreau Envelope Smoothing for Nonsmooth Nonconvex Problems
Antonio Silveti-Falls, Cesare Molinari, Zev Woodstock
We present and analyze Frank-Wolfe with Moreau Envelope Smoothing (FRAMES) for solving nonsmooth nonconvex constrained optimization problems, taking advantage of iterative smoothin…
Boosted Stochastic Frank-Wolfe for Constrained Nonconvex Optimization
Navil Nandhan, Abbas Khademi, Antonio Silveti-Falls
The boosted Frank-Wolfe algorithm accelerates the classical Frank-Wolfe algorithm by better aligning the update direction with the negative gradient. Its analysis, however, has bee…
Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
Konstantinos Oikonomidis, Jan Quan, Kimon Antonakopoulos +3
In this work, we develop proximal preconditioned gradient methods with a focus on spectral gradient methods providing a proximal extension to the Muon and Scion optimizers. We intr…
On the Role of Batch Size in Stochastic Conditional Gradient Methods
Rustem Islamov, Roman Machacek, Aurelien Lucchi +3
We study the role of batch size in stochastic conditional gradient methods under a -Kurdyka-Łojasiewicz (-KL) condition. Focusing on momentum-based stochastic conditional gra…
Training Neural Networks at Any Scale
Thomas Pethick, Kimon Antonakopoulos, Antonio Silveti-Falls +2
This article reviews modern optimization methods for training neural networks with an emphasis on efficiency and scale. We present state-of-the-art optimization algorithms under a…
Adaptive Conditional Gradient Descent
Abbas Khademi, Antonio Silveti-Falls
Selecting an effective step-size is a fundamental challenge in first-order optimization, especially for problems with non-Euclidean geometries. This paper presents a novel adaptive…
Generalized Gradient Norm Clipping & Non-Euclidean -Smoothness
Thomas Pethick, Wanyun Xie, Mete Erdogan +3
This work introduces a hybrid non-Euclidean optimization method which generalizes gradient norm clipping by combining steepest descent and conditional gradient approaches. The meth…
Training Deep Learning Models with Norm-Constrained LMOs
Thomas Pethick, Wanyun Xie, Kimon Antonakopoulos +3
In this work, we study optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball. We propose a new stochastic family of algorithms that uses the LMO…
Differentiating Nonsmooth Solutions to Parametric Monotone Inclusion Problems
Jérôme Bolte, Edouard Pauwels, Antonio Silveti-Falls
We leverage path differentiability and a recent result on nonsmooth implicit differentiation calculus to give sufficient conditions ensuring that the solution to a monotone inclusi…
A Stochastic Bregman Primal-Dual Splitting Algorithm for Composite Optimization
Antonio Silveti-Falls, Cesare Molinari, Jalal Fadili
We study a stochastic first order primal-dual method for solving convex-concave saddle point problems over real reflexive Banach spaces using Bregman divergences and relative smoot…
Nonsmooth Implicit Differentiation for Machine Learning and Optimization
Jérôme Bolte, Tam Le, Edouard Pauwels +1
In view of training increasingly complex learning architectures, we establish a nonsmooth implicit function theorem with an operational calculus. Our result applies to most practic…
Inexact and Stochastic Generalized Conditional Gradient with Augmented Lagrangian and Proximal Step
Antonio Silveti-Falls, Cesare Molinari, Jalal Fadili
In this paper we propose and analyze inexact and stochastic versions of the CGALP algorithm developed in the authors' previous paper, which we denote ICGALP, that allows for errors…
Generalized Conditional Gradient with Augmented Lagrangian for Composite Minimization
Antonio Silveti-Falls, Cesare Molinari, Jalal Fadili
In this paper we propose a splitting scheme which hybridizes generalized conditional gradient with a proximal step which we call CGALP algorithm, for minimizing the sum of three pr…