78 citations · 172 across the 12 of their papers we have counts for
6 papers · 1 filter
A Study of Condition Numbers for First-Order Optimization
Charles Guille-Escuret, Baptiste Goujaud, Manuela Girotti +1
The study of first-order optimization algorithms (FOA) typically starts with assumptions on the objective functions, most commonly smoothness and strong convexity. These metrics ar…
Adversarial score matching and improved sampling for image generation
Alexia Jolicoeur-Martineau, Rémi Piché-Taillefer, Rémi Tachet des Combes +1
Denoising Score Matching with Annealed Langevin Sampling (DSM-ALS) has recently found success in generative modeling. The approach works by first training a neural network to estim…
LEAD: Min-Max Optimization from a Physical Perspective
Reyhane Askari Hemmat, Amartya Mitra, Guillaume Lajoie +1
Adversarial formulations such as generative adversarial networks (GANs) have rekindled interest in two-player min-max games. A central obstacle in the optimization of such games is…
In Search of Robust Measures of Generalization
Gintare Karolina Dziugaite, Alexandre Drouin, Brady Neal +5
One of the principal scientific challenges in deep learning is explaining generalization, i.e., why the particular way the community now trains networks to achieve small training e…
Stochastic Hamiltonian Gradient Methods for Smooth Games
Nicolas Loizou, Hugo Berard, Alexia Jolicoeur-Martineau +3
The success of adversarial formulations in machine learning has brought renewed motivation for smooth games. In this work, we focus on the class of stochastic Hamiltonian methods a…
Accelerating Smooth Games by Manipulating Spectral Shapes
Waïss Azizian, Damien Scieur, Ioannis Mitliagkas +2
We use matrix iteration theory to characterize acceleration in smooth games. We define the spectral shape of a family of games as the set containing all eigenvalues of the Jacobian…