503 citations · 1.2k across the 17 of their papers we have counts for
8 papers · 1 filter
Why neural networks find simple solutions: the many regularizers of geometric complexity
Benoit Dherin, Michael Munn, Mihaela Rosca +1
In many contexts, simpler models are preferable to more complex models and the control of this model complexity is the goal for many methods in machine learning such as regularizat…
Privacy-Preserving Serverless Edge Learning with Decentralized Small Data
Shih-Chun Lin, Chia-Hung Lin
In the last decade, data-driven algorithms outperformed traditional optimization-based algorithms in many research areas, such as computer vision, natural language processing, etc.…
On the Origin of Implicit Regularization in Stochastic Gradient Descent
Samuel L. Smith, Benoit Dherin, David G. T. Barrett +1
For infinitesimal learning rates, stochastic gradient descent (SGD) follows the path of gradient flow on the full batch loss function. However moderately large learning rates can a…
Implicit Gradient Regularization
David G. T. Barrett, Benoit Dherin
Gradient descent can be surprisingly good at optimizing deep neural networks without overfitting and without explicit regularization. We find that the discrete steps of gradient de…
An Explicitly Relational Neural Network Architecture
Murray Shanahan, Kyriacos Nikiforou, Antonia Creswell +3
With a view to bridging the gap between deep learning and symbolic AI, we present a novel end-to-end neural network architecture that learns to form propositional representations w…
Measuring abstract reasoning in neural networks
David G. T. Barrett, Felix Hill, Adam Santoro +2
Whether neural networks can learn abstract reasoning or whether they merely rely on superficial statistics is a topic of recent debate. Here, we propose a dataset and challenge des…
Spectral Inference Networks: Unifying Deep and Spectral Learning
David Pfau, Stig Petersen, Ashish Agarwal +2
We present Spectral Inference Networks, a framework for learning eigenfunctions of linear operators by stochastic optimization. Spectral Inference Networks generalize Slow Feature…
Discovering objects and their relations from entangled scene representations
David Raposo, Adam Santoro, David Barrett +3
Our world can be succinctly and compactly described as structured scenes of objects and relations. A typical room, for example, contains salient objects such as tables, chairs and…