503 citations · 1.2k across the 17 of their papers we have counts for
19 papers
Why neural networks find simple solutions: the many regularizers of geometric complexity
Benoit Dherin, Michael Munn, Mihaela Rosca +1
In many contexts, simpler models are preferable to more complex models and the control of this model complexity is the goal for many methods in machine learning such as regularizat…
Privacy-Preserving Serverless Edge Learning with Decentralized Small Data
Shih-Chun Lin, Chia-Hung Lin
In the last decade, data-driven algorithms outperformed traditional optimization-based algorithms in many research areas, such as computer vision, natural language processing, etc.…
Discretization Drift in Two-Player Games
Mihaela Rosca, Yan Wu, Benoit Dherin +1
Gradient-based methods for two-player games produce rich dynamics that can solve challenging problems, yet can be difficult to stabilize and understand. Part of this complexity ori…
On the Origin of Implicit Regularization in Stochastic Gradient Descent
Samuel L. Smith, Benoit Dherin, David G. T. Barrett +1
For infinitesimal learning rates, stochastic gradient descent (SGD) follows the path of gradient flow on the full batch loss function. However moderately large learning rates can a…
Implicit Gradient Regularization
David G. T. Barrett, Benoit Dherin
Gradient descent can be surprisingly good at optimizing deep neural networks without overfitting and without explicit regularization. We find that the discrete steps of gradient de…
What can the brain teach us about building artificial intelligence?
Dileep George
This paper is the preprint of an invited commentary on Lake et al's Behavioral and Brain Sciences article titled "Building machines that learn and think like people". Lake et al's…