3 papers
cs.LG2026
Navigating Potholes with Geometry-Aware Sharpness Minimization
Simon Dufort-Labbé, Mehrab Hamidi, Razvan Pascanu +3
Sharpness-aware minimization (SAM) encourages flat minima by perturbing parameters along directions of high loss curvature, but treats all parameter directions uniformly, ignoring…
cs.LG2026
Layerwise LQR for Geometry-Aware Optimization of Deep Networks
Simon Dufort-Labbé, Pierre-Luc Bacon, Razvan Pascanu +2
Geometry-aware optimizers such as Newton and natural gradient can improve conditioning in deep learning, but scalable variants such as K-FAC, Shampoo, and related preconditioners u…
cs.LG2025
Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neurons
Simon Dufort-Labbé, Pierluca D'Oro, Evgenii Nikishin +3
When training neural networks, dying neurons -- units becoming inactive or saturated -- are traditionally seen as harmful. This paper sheds new light on this phenomenon. By explori…