3 papers
cs.LG2025
When majority rules, minority loses: bias amplification of gradient descent
François Bachoc, Jérôme Bolte, Ryan Boustany +1
Despite growing empirical evidence of bias amplification in machine learning, its theoretical foundations remain poorly understood. We develop a formal framework for majority-minor…
cs.LG2024
A second-order-like optimizer with adaptive gradient scaling for deep learning
Jérôme Bolte, Ryan Boustany, Edouard Pauwels +1
In this empirical article, we introduce INNAprop, an optimization algorithm that combines the INNA method with the RMSprop adaptive gradient scaling. It leverages second-order info…
cs.LG2024
On the numerical reliability of nonsmooth autodiff: a MaxPool case study
Ryan Boustany
This paper considers the reliability of automatic differentiation (AD) for neural networks involving the nonsmooth MaxPool operation. We investigate the behavior of AD across diffe…