22k citations · 25.1k across the 40 of their papers we have counts for
13 papers · 1 filter
Big Transfer (BiT): General Visual Representation Learning
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai +4
Transfer of pre-trained representations improves sample efficiency and simplifies hyperparameter tuning when training deep neural networks for vision. We revisit the paradigm of pr…
Self-Supervised Learning of Video-Induced Visual Invariances
Michael Tschannen, Josip Djolonga, Marvin Ritter +5
We propose a general framework for self-supervised learning of transferable visual representations based on Video-Induced Visual Invariances (VIVI). We consider the implicit hierar…
Semantic Bottleneck Scene Generation
Samaneh Azadi, Michael Tschannen, Eric Tzeng +3
Coupling the high-fidelity generation capabilities of label-conditional image synthesis methods with the flexibility of unconditional generative models, we propose a semantic bottl…
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov +14
Representation learning promises to unlock deep learning for the long tail of vision tasks without expensive labelled datasets. Yet, the absence of a unified evaluation for general…
On Mutual Information Maximization for Representation Learning
Michael Tschannen, Josip Djolonga, Paul K. Rubenstein +2
Many recent methods for unsupervised or self-supervised representation learning train feature extractors by maximizing an estimate of the mutual information (MI) between different…
Google Research Football: A Novel Reinforcement Learning Environment
Karol Kurach, Anton Raichuk, Piotr Stańczyk +8
Recent progress in the field of reinforcement learning has been accelerated by virtual learning environments such as video games, where novel algorithms and ideas can be quickly te…
MULEX: Disentangling Exploitation from Exploration in Deep RL
Lucas Beyer, Damien Vincent, Olivier Teboul +3
An agent learning through interactions should balance its action selection process between probing the environment to discover new rewards and using the information acquired in the…
Adaptive Temporal-Difference Learning for Policy Evaluation with Per-State Uncertainty Estimates
Hugo Penedones, Carlos Riquelme, Damien Vincent +5
We consider the core reinforcement-learning problem of on-policy value function approximation from a batch of trajectory data, and focus on various issues of Temporal Difference (T…
When can unlabeled data improve the learning rate?
Christina Göpfert, Shai Ben-David, Olivier Bousquet +3
In semi-supervised classification, one is given access both to labeled and unlabeled data. As unlabeled data is typically cheaper to acquire than labeled data, this setup becomes a…
Precision-Recall Curves Using Information Divergence Frontiers
Josip Djolonga, Mario Lucic, Marco Cuturi +3
Despite the tremendous progress in the estimation of generative models, the development of tools for diagnosing their failures and assessing their performance has advanced at a muc…
High-Fidelity Image Generation With Fewer Labels
Mario Lucic, Michael Tschannen, Marvin Ritter +3
Deep generative models are becoming a cornerstone of modern machine learning. Recent work on conditional generative adversarial networks has shown that learning complex, high-dimen…
Breaking the Softmax Bottleneck via Learnable Monotonic Pointwise Non-linearities
Octavian-Eugen Ganea, Sylvain Gelly, Gary Bécigneul +1
The Softmax function on top of a final linear layer is the de facto method to output probability distributions in neural networks. In many applications such as language models or t…
Parameter-Efficient Transfer Learning for NLP
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski +5
Fine-tuning large pre-trained models is an effective transfer mechanism in NLP. However, in the presence of many downstream tasks, fine-tuning is parameter inefficient: an entire n…