1 citations · 1 across the 7 of their papers we have counts for
9 papers
On the Comparison of Optimizers for Imbalanced Learning
Cl{é}ment Lezane, Fran{\c c}ois Bachoc, J{é}r{ô}me Bolte +1
Data imbalance is pervasive in machine learning, from rare words and anomalies to underrepresented patterns in heterogeneous or cross-tabulated data. We study idealized optimizers…
The adjoint state method for parametric definable optimization without smoothness or uniqueness
Jérôme Bolte, Edouard Pauwels, Cheik Traoré
We establish that nonconvex definable parametric optimization problems with possibly nonsmooth objectives, inequality constraints, conic constraint systems, and non-unique primal a…
Convergence of optimizers implies eigenvalues filtering at equilibrium
Jerome Bolte, Quoc-Tung Le, Edouard Pauwels
Ample empirical evidence in deep neural network training suggests that a variety of optimizers tend to find nearly global optima. In this article, we adopt the reversed perspective…
When majority rules, minority loses: bias amplification of gradient descent
François Bachoc, Jérôme Bolte, Ryan Boustany +1
Despite growing empirical evidence of bias amplification in machine learning, its theoretical foundations remain poorly understood. We develop a formal framework for majority-minor…
Bilevel gradient methods and the Morse parametric qualification condition
Jérôme Bolte, Quoc-Tung Le, Edouard Pauwels +1
We introduce the Morse parametric qualification condition for bilevel programming. Generic semi-algebraic functions are Morse parametric in a piecewise sense. Thus, bilevel program…
A second-order-like optimizer with adaptive gradient scaling for deep learning
Jérôme Bolte, Ryan Boustany, Edouard Pauwels +1
In this empirical article, we introduce INNAprop, an optimization algorithm that combines the INNA method with the RMSprop adaptive gradient scaling. It leverages second-order info…
Geometric and computational hardness of bilevel programming
Jérôme Bolte, Quoc-Tung Le, Edouard Pauwels +1
We first show a simple but striking result in bilevel optimization: unconstrained smooth bilevel programming is as hard as general extended-real-valued lower semicontinu…
Inexact subgradient methods for semialgebraic functions
Jérôme Bolte, Tam Le, Éric Moulines +1
Motivated by the extensive application of approximate gradients in machine learning and optimization, we investigate inexact subgradient methods subject to persistent additive erro…
Numerical influence of ReLU'(0) on backpropagation
David Bertoin, Jérôme Bolte, Sébastien Gerchinovitz +1
In theory, the choice of ReLU(0) in [0, 1] for a neural network has a negligible influence both on backpropagation and training. Yet, in the real world, 32 bits default precision c…