78 citations · 165 across the 10 of their papers we have counts for
22 papers
Convergence Analysis and Implicit Regularization of Feedback Alignment for Deep Linear Networks
Manuela Girotti, Ioannis Mitliagkas, Gauthier Gidel
We theoretically analyze the Feedback Alignment (FA) algorithm, an efficient alternative to backpropagation for training neural networks. We provide convergence guarantees with rat…
Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity
Nicolas Loizou, Hugo Berard, Gauthier Gidel +2
Two of the most prominent algorithms for solving unconstrained smooth games are the classical stochastic gradient descent-ascent (SGDA) and the recently introduced stochastic conse…
Adversarial score matching and improved sampling for image generation
Alexia Jolicoeur-Martineau, Rémi Piché-Taillefer, Rémi Tachet des Combes +1
Denoising Score Matching with Annealed Langevin Sampling (DSM-ALS) has recently found success in generative modeling. The approach works by first training a neural network to estim…
In Search of Robust Measures of Generalization
Gintare Karolina Dziugaite, Alexandre Drouin, Brady Neal +5
One of the principal scientific challenges in deep learning is explaining generalization, i.e., why the particular way the community now trains networks to achieve small training e…
Stochastic Hamiltonian Gradient Methods for Smooth Games
Nicolas Loizou, Hugo Berard, Alexia Jolicoeur-Martineau +3
The success of adversarial formulations in machine learning has brought renewed motivation for smooth games. In this work, we focus on the class of stochastic Hamiltonian methods a…
Accelerating Smooth Games by Manipulating Spectral Shapes
Waïss Azizian, Damien Scieur, Ioannis Mitliagkas +2
We use matrix iteration theory to characterize acceleration in smooth games. We define the spectral shape of a family of games as the set containing all eigenvalues of the Jacobian…