12 papers
Accelerated and Stable Convergence with Anchored Generalized Optimistic Method
Motahareh Sohrabi, Jianxin You, Simon Lacoste-Julien +2
We study first-order methods for solving monotone variational inequalities arising in min-max optimization. Classical approaches such as the extragradient method rely on two gradie…
Unsupervised Causal Abstractions Discovery
Théo Saulus, Simon Lacoste-Julien, Dhanya Sridhar
Causal abstractions formalize when a high-level structural causal model (SCM) captures the interventional behavior of a lower-level SCM. Existing applications of this notion largel…
The Role of Causal Features in Strategic Classification for Robustness and Alignment
Antonio Gois, Sophia Gunluk, Nir Rosenfeld +3
In strategic classification, an institution (e.g., a bank) anticipates adaptation from users who change their features to increase utility in a classification task (e.g., loan repa…
Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage
Alan Milligan, Zikun Xu, Simon Lacoste-Julien +2
Shampoo-based methods, such as KL-Shampoo and SOAP, have demonstrated strong performance in training neural networks and rely on QR decomposition. Because existing QR implementatio…
Position: Adopt Constraints Over Fixed Penalties in Deep Learning
Juan Ramirez, Meraj Hashemizadeh, Simon Lacoste-Julien
Recent efforts to develop trustworthy AI systems have increased interest in learning problems with explicit requirements, or constraints. In deep learning, however, such problems a…
Layerwise LQR for Geometry-Aware Optimization of Deep Networks
Simon Dufort-Labbé, Pierre-Luc Bacon, Razvan Pascanu +2
Geometry-aware optimizers such as Newton and natural gradient can improve conditioning in deep learning, but scalable variants such as K-FAC, Shampoo, and related preconditioners u…