110 citations · 141 across the 5 of their papers we have counts for
5 papers
Double-descent curves in neural networks: a new perspective using Gaussian processes
Ouns El Harzli, Bernardo Cuenca Grau, Guillermo Valle-Pérez +1
Double-descent curves in neural networks describe the phenomenon that the generalisation error initially descends with increasing parameters, then grows after reaching an optimal n…
Generalization bounds for deep learning
Guillermo Valle-Pérez, Ard A. Louis
Generalization in deep learning has been the topic of much recent theoretical and empirical research. Here we introduce desiderata for techniques that predict generalization errors…
Is SGD a Bayesian sampler? Well, almost
Chris Mingard, Guillermo Valle-Pérez, Joar Skalse +1
Overparameterised deep neural networks (DNNs) are highly expressive and so can, in principle, generate almost any function that fits a training dataset with zero error. The vast ma…
Neural networks are a priori biased towards Boolean functions with low entropy
Chris Mingard, Joar Skalse, Guillermo Valle-Pérez +3
Understanding the inductive bias of neural networks is critical to explaining their ability to generalise. Here, for one of the simplest neural networks -- a single-layer perceptro…
Deep learning generalizes because the parameter-function map is biased towards simple functions
Guillermo Valle-Pérez, Chico Q. Camargo, Ard A. Louis
Deep neural networks (DNNs) generalize remarkably well without explicit regularization even in the strongly over-parametrized regime where classical learning theory would instead p…