2k citations · 4.8k across the 35 of their papers we have counts for
4 papers · 1 filter
Stochastic Gradient Push for Distributed Deep Learning
Mahmoud Assran, Nicolas Loizou, Nicolas Ballas +1
Distributed data-parallel algorithms aim to accelerate the training of deep neural networks by parallelizing the computation of large mini-batch gradient updates across multiple no…
On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length
Stanisław Jastrzębski, Zachary Kenton, Nicolas Ballas +3
Stochastic Gradient Descent (SGD) based training of neural networks with a large learning rate or a small batch-size typically ends in well-generalizing, flat regions of the weight…
A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
Amy Zhang, Nicolas Ballas, Joelle Pineau
The risks and perils of overfitting in machine learning are well known. However most of the treatment of this, including diagnostic tools and remedies, was developed for the superv…
Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis
Thomas George, César Laurent, Xavier Bouthillier +2
Optimization algorithms that leverage gradient covariance information, such as variants of natural gradient descent (Amari, 1998), offer the prospect of yielding more effective des…