22k citations · 26.7k across the 60 of their papers we have counts for
8 papers · 1 filter
Underspecification Presents Challenges for Credibility in Modern Machine Learning
Alexander D'Amour, Katherine Heller, Dan Moldovan +37
ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline i…
Deep Ensembles for Low-Data Transfer Learning
Basil Mustafa, Carlos Riquelme, Joan Puigcerver +3
In the low-data regime, it is difficult to train good supervised models from scratch. Instead practitioners turn to pre-trained models, leveraging transfer learning. Ensembling is…
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov +9
While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention…
Representation learning from videos in-the-wild: An object-centric approach
Rob Romijnders, Aravindh Mahendran, Michael Tschannen +4
We propose a method to learn image representations from uncurated videos. We combine a supervised loss from off-the-shelf object detectors and self-supervised losses which naturall…
Training general representations for remote sensing using in-domain knowledge
Maxim Neumann, André Susano Pinto, Xiaohua Zhai +1
Automatically finding good and general remote sensing representations allows to perform transfer learning on a wide range of applications - improving the accuracy and reducing the…
Scalable Transfer Learning with Expert Models
Joan Puigcerver, Carlos Riquelme, Basil Mustafa +5
Transfer of pre-trained representations can improve sample efficiency and reduce computational requirements for new tasks. However, representations used for transfer are usually ge…
On Robustness and Transferability of Convolutional Neural Networks
Josip Djolonga, Jessica Yung, Michael Tschannen +11
Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts. However, several recent breakthroughs in transfer learning suggest…
Automatic Shortcut Removal for Self-Supervised Representation Learning
Matthias Minderer, Olivier Bachem, Neil Houlsby +1
In self-supervised visual representation learning, a feature extractor is trained on a "pretext task" for which labels can be generated cheaply, without human annotation. A central…