22k citations · 25.1k across the 40 of their papers we have counts for
9 papers · 1 filter
A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation
Francesco Locatello, Stefan Bauer, Mario Lucic +4
The idea behind the \emph{unsupervised} learning of \emph{disentangled} representations is that real-world data is generated by a few explanatory factors of variation which can be…
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov +9
While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention…
Scalable Transfer Learning with Expert Models
Joan Puigcerver, Carlos Riquelme, Basil Mustafa +5
Transfer of pre-trained representations can improve sample efficiency and reduce computational requirements for new tasks. However, representations used for transfer are usually ge…
A Commentary on the Unsupervised Learning of Disentangled Representations
Francesco Locatello, Stefan Bauer, Mario Lucic +4
The goal of the unsupervised learning of disentangled representations is to separate the independent explanatory factors of variation in the data without access to supervision. In…
On Robustness and Transferability of Convolutional Neural Networks
Josip Djolonga, Jessica Yung, Michael Tschannen +11
Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts. However, several recent breakthroughs in transfer learning suggest…
What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study
Marcin Andrychowicz, Anton Raichuk, Piotr Stańczyk +9
In recent years, on-policy reinforcement learning (RL) has been successfully applied to many different continuous control tasks. While RL algorithms are often conceptually simple,…
What Do Neural Networks Learn When Trained With Random Labels?
Hartmut Maennel, Ibrahim Alabdulmohsin, Ilya Tolstikhin +4
We study deep neural networks (DNNs) trained on natural image data with entirely random labels. Despite its popularity in the literature, where it is often used to study memorizati…
Predicting Neural Network Accuracy from Weights
Thomas Unterthiner, Daniel Keysers, Sylvain Gelly +2
We show experimentally that the accuracy of a trained neural network can be predicted surprisingly well by looking only at its weights, without evaluating it on input data. We moti…
On Last-Layer Algorithms for Classification: Decoupling Representation from Uncertainty Estimation
Nicolas Brosse, Carlos Riquelme, Alice Martin +2
Uncertainty quantification for deep learning is a challenging open problem. Bayesian statistics offer a mathematically grounded framework to reason about uncertainties; however, ap…