14 citations · 42 across the 11 of their papers we have counts for
11 papers
A simple mean field model of feature learning
Niclas Göring, Chris Mingard, Yoonsoo Nam +1
Feature learning (FL), where neural networks adapt their internal representations during training, remains poorly understood. Using methods from statistical physics, we derive a tr…
Feature learning is decoupled from generalization in high capacity neural networks
Niclas Alexander Göring, Charles London, Abdurrahman Hadi Erturk +3
Neural networks outperform kernel methods, sometimes by orders of magnitude, e.g. on staircase functions. This advantage stems from the ability of neural networks to learn features…
Characterising the Inductive Biases of Neural Networks on Boolean Data
Chris Mingard, Lukas Seier, Niclas Göring +3
Deep neural networks are renowned for their ability to generalise well across diverse tasks, even when heavily overparameterized. Existing works offer only partial explanations (fo…
Decoupling Dynamical Richness from Representation Learning: Towards Practical Measurement
Yoonsoo Nam, Nayara Fonseca, Seok Hyeong Lee +6
Dynamic feature transformation (the rich regime) does not always align with predictive performance (better representation), yet accuracy is often used as a proxy for richness, limi…
Exploiting the equivalence between quantum neural networks and perceptrons
Chris Mingard, Jessica Pointing, Charles London +2
Quantum machine learning models based on parametrized quantum circuits, also called quantum neural networks (QNNs), are considered to be among the most promising candidates for app…
An exactly solvable model for emergence and scaling laws in the multitask sparse parity problem
Yoonsoo Nam, Nayara Fonseca, Seok Hyeong Lee +2
Deep learning models can exhibit what appears to be a sudden ability to solve a new problem as training time, training data, or model size increases, a phenomenon known as emergenc…
Automatic Gradient Descent: Deep Learning without Hyperparameters
Jeremy Bernstein, Chris Mingard, Kevin Huang +2
The architecture of a deep neural network is defined explicitly in terms of the number of layers, the width of each layer and the general network topology. Existing optimisation fr…
Deep neural networks have an inbuilt Occam's razor
Chris Mingard, Henry Rees, Guillermo Valle-Pérez +1
The remarkable performance of overparameterized deep neural networks (DNNs) must arise from an interplay between network architecture, training algorithms, and structure in the dat…
Feature Learning and Signal Propagation in Deep Neural Networks
Yizhang Lou, Chris Mingard, Yoonsoo Nam +1
Recent work by Baratin et al. (2021) sheds light on an intriguing pattern that occurs during the training of deep neural networks: some layers align much more with data compared to…
Is SGD a Bayesian sampler? Well, almost
Chris Mingard, Guillermo Valle-Pérez, Joar Skalse +1
Overparameterised deep neural networks (DNNs) are highly expressive and so can, in principle, generate almost any function that fits a training dataset with zero error. The vast ma…
Neural networks are a priori biased towards Boolean functions with low entropy
Chris Mingard, Joar Skalse, Guillermo Valle-Pérez +3
Understanding the inductive bias of neural networks is critical to explaining their ability to generalise. Here, for one of the simplest neural networks -- a single-layer perceptro…