927 citations · 2.7k across the 88 of their papers we have counts for
10 papers · 1 filter
Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation
Si Yi Meng, Sharan Vaswani, Issam Laradji +2
We consider stochastic second-order methods for minimizing smooth and strongly-convex functions under an interpolation condition satisfied by over-parameterized models. Under this…
GAIT: A Geometric Approach to Information Theory
Jose Gallego-Posada, Ankit Vani, Max Schwarzer +1
We advocate the use of a notion of entropy that reflects the relative abundances of the symbols in an alphabet, as well as the similarities between them. This concept was originall…
A Tight and Unified Analysis of Gradient-Based Methods for a Whole Spectrum of Games
Waïss Azizian, Ioannis Mitliagkas, Simon Lacoste-Julien +1
We consider differentiable games where the goal is to find a Nash equilibrium. The machine learning community has recently started using variants of the gradient method (GD). Prime…
A Closer Look at the Optimization Landscapes of Generative Adversarial Networks
Hugo Berard, Gauthier Gidel, Amjad Almahairi +2
Generative adversarial networks have been very successful in generative modeling, however they remain relatively challenging to train compared to standard deep neural networks. In…
Gradient-Based Neural DAG Learning
Sébastien Lachapelle, Philippe Brouillard, Tristan Deleu +1
We propose a novel score-based approach to learning a directed acyclic graph (DAG) from observational data. We adapt a recently proposed continuous constrained optimization formula…
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates
Sharan Vaswani, Aaron Mishkin, Issam Laradji +3
Recent works have shown that stochastic gradient descent (SGD) achieves the fast convergence rates of full-batch gradient descent for over-parameterized models satisfying certain i…