149 citations · 453 across the 21 of their papers we have counts for
7 papers · 1 filter
The twin peaks of learning neural networks
Elizaveta Demyanenko, Christoph Feinauer, Enrico M. Malatesta +1
Recent works demonstrated the existence of a double-descent phenomenon for the generalization error of neural networks, where highly overparameterized models escape overfitting and…
Optimal transfer protocol by incremental layer defrosting
Federica Gerace, Diego Doimo, Stefano Sarao Mannelli +2
Transfer learning is a powerful tool enabling model training with limited amounts of data. This technique is particularly useful in real-world problems where data availability is o…
Bias-inducing geometries: an exactly solvable data model with fairness implications
Stefano Sarao Mannelli, Federica Gerace, Negar Rostamzadeh +1
Machine learning (ML) may be oblivious to human bias but it is not immune to its perpetuation. Marginalisation and iniquitous group representation are often traceable in the very d…
An Analytical Theory of Curriculum Learning in Teacher-Student Networks
Luca Saglietti, Stefano Sarao Mannelli, Andrew Saxe
In humans and animals, curriculum learning -- presenting data in a curated order - is critical to rapid learning and effective pedagogy. Yet in machine learning, curricula are not…
Probing transfer learning with a model of synthetic correlated datasets
Federica Gerace, Luca Saglietti, Stefano Sarao Mannelli +2
Transfer learning can significantly improve the sample efficiency of neural networks, by exploiting the relatedness between a data-scarce target task and a data-abundant source tas…
Solvable Model for Inheriting the Regularization through Knowledge Distillation
Luca Saglietti, Lenka Zdeborová
In recent years the empirical success of transfer learning with neural networks has stimulated an increasing interest in obtaining a theoretical understanding of its core propertie…
Gaussian Process Prior Variational Autoencoders
Francesco Paolo Casale, Adrian V Dalca, Luca Saglietti +2
Variational autoencoders (VAE) are a powerful and widely-used class of models to learn complex data distributions in an unsupervised fashion. One important limitation of VAEs is th…