5 papers · 1 filter
Sufficient Conditions for Stability of Minimum-Norm Interpolating Deep ReLU Networks
Ouns El Harzli, Yoonsoo Nam, Ilja Kuzborskij +2
Algorithmic stability is a classical framework for analyzing the generalization error of learning algorithms. It predicts that an algorithm has small generalization error if it is…
A simple mean field model of feature learning
Niclas Göring, Chris Mingard, Yoonsoo Nam +1
Feature learning (FL), where neural networks adapt their internal representations during training, remains poorly understood. Using methods from statistical physics, we derive a tr…
Feature learning is decoupled from generalization in high capacity neural networks
Niclas Alexander Göring, Charles London, Abdurrahman Hadi Erturk +3
Neural networks outperform kernel methods, sometimes by orders of magnitude, e.g. on staircase functions. This advantage stems from the ability of neural networks to learn features…
Characterising the Inductive Biases of Neural Networks on Boolean Data
Chris Mingard, Lukas Seier, Niclas Göring +3
Deep neural networks are renowned for their ability to generalise well across diverse tasks, even when heavily overparameterized. Existing works offer only partial explanations (fo…
An exactly solvable model for emergence and scaling laws in the multitask sparse parity problem
Yoonsoo Nam, Nayara Fonseca, Seok Hyeong Lee +2
Deep learning models can exhibit what appears to be a sudden ability to solve a new problem as training time, training data, or model size increases, a phenomenon known as emergenc…