1.4k citations · 1.7k across the 11 of their papers we have counts for
4 papers · 1 filter
Deep Ensembles for Low-Data Transfer Learning
Basil Mustafa, Carlos Riquelme, Joan Puigcerver +3
In the low-data regime, it is difficult to train good supervised models from scratch. Instead practitioners turn to pre-trained models, leveraging transfer learning. Ensembling is…
Scalable Transfer Learning with Expert Models
Joan Puigcerver, Carlos Riquelme, Basil Mustafa +5
Transfer of pre-trained representations can improve sample efficiency and reduce computational requirements for new tasks. However, representations used for transfer are usually ge…
What Do Neural Networks Learn When Trained With Random Labels?
Hartmut Maennel, Ibrahim Alabdulmohsin, Ilya Tolstikhin +4
We study deep neural networks (DNNs) trained on natural image data with entirely random labels. Despite its popularity in the literature, where it is often used to study memorizati…
Predicting Neural Network Accuracy from Weights
Thomas Unterthiner, Daniel Keysers, Sylvain Gelly +2
We show experimentally that the accuracy of a trained neural network can be predicted surprisingly well by looking only at its weights, without evaluating it on input data. We moti…