5 papers
The adjoint state method for parametric definable optimization without smoothness or uniqueness
Jérôme Bolte, Edouard Pauwels, Cheik Traoré
We establish that nonconvex definable parametric optimization problems with possibly nonsmooth objectives, inequality constraints, conic constraint systems, and non-unique primal a…
Convergence of optimizers implies eigenvalues filtering at equilibrium
Jerome Bolte, Quoc-Tung Le, Edouard Pauwels
Ample empirical evidence in deep neural network training suggests that a variety of optimizers tend to find nearly global optima. In this article, we adopt the reversed perspective…
When majority rules, minority loses: bias amplification of gradient descent
François Bachoc, Jérôme Bolte, Ryan Boustany +1
Despite growing empirical evidence of bias amplification in machine learning, its theoretical foundations remain poorly understood. We develop a formal framework for majority-minor…
Bilevel gradient methods and the Morse parametric qualification condition
Jérôme Bolte, Quoc-Tung Le, Edouard Pauwels +1
We introduce the Morse parametric qualification condition for bilevel programming. Generic semi-algebraic functions are Morse parametric in a piecewise sense. Thus, bilevel program…
A second-order-like optimizer with adaptive gradient scaling for deep learning
Jérôme Bolte, Ryan Boustany, Edouard Pauwels +1
In this empirical article, we introduce INNAprop, an optimization algorithm that combines the INNA method with the RMSprop adaptive gradient scaling. It leverages second-order info…