22k citations · 25.1k across the 40 of their papers we have counts for
24 papers · 1 filter
Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark
Vincent Dumoulin, Neil Houlsby, Utku Evci +4
Meta and transfer learning are two successful families of approaches to few-shot learning. Despite highly related goals, state-of-the-art advances in each family are measured large…
A Sober Look at the Unsupervised Learning of Disentangled Representations and their Evaluation
Francesco Locatello, Stefan Bauer, Mario Lucic +4
The idea behind the \emph{unsupervised} learning of \emph{disentangled} representations is that real-world data is generated by a few explanatory factors of variation which can be…
Scalable Transfer Learning with Expert Models
Joan Puigcerver, Carlos Riquelme, Basil Mustafa +5
Transfer of pre-trained representations can improve sample efficiency and reduce computational requirements for new tasks. However, representations used for transfer are usually ge…
A Commentary on the Unsupervised Learning of Disentangled Representations
Francesco Locatello, Stefan Bauer, Mario Lucic +4
The goal of the unsupervised learning of disentangled representations is to separate the independent explanatory factors of variation in the data without access to supervision. In…
What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study
Marcin Andrychowicz, Anton Raichuk, Piotr Stańczyk +9
In recent years, on-policy reinforcement learning (RL) has been successfully applied to many different continuous control tasks. While RL algorithms are often conceptually simple,…
Semantic Bottleneck Scene Generation
Samaneh Azadi, Michael Tschannen, Eric Tzeng +3
Coupling the high-fidelity generation capabilities of label-conditional image synthesis methods with the flexibility of unconditional generative models, we propose a semantic bottl…
On Mutual Information Maximization for Representation Learning
Michael Tschannen, Josip Djolonga, Paul K. Rubenstein +2
Many recent methods for unsupervised or self-supervised representation learning train feature extractors by maximizing an estimate of the mutual information (MI) between different…
Google Research Football: A Novel Reinforcement Learning Environment
Karol Kurach, Anton Raichuk, Piotr Stańczyk +8
Recent progress in the field of reinforcement learning has been accelerated by virtual learning environments such as video games, where novel algorithms and ideas can be quickly te…
MULEX: Disentangling Exploitation from Exploration in Deep RL
Lucas Beyer, Damien Vincent, Olivier Teboul +3
An agent learning through interactions should balance its action selection process between probing the environment to discover new rewards and using the information acquired in the…
Adaptive Temporal-Difference Learning for Policy Evaluation with Per-State Uncertainty Estimates
Hugo Penedones, Carlos Riquelme, Damien Vincent +5
We consider the core reinforcement-learning problem of on-policy value function approximation from a batch of trajectory data, and focus on various issues of Temporal Difference (T…
When can unlabeled data improve the learning rate?
Christina Göpfert, Shai Ben-David, Olivier Bousquet +3
In semi-supervised classification, one is given access both to labeled and unlabeled data. As unlabeled data is typically cheaper to acquire than labeled data, this setup becomes a…
Precision-Recall Curves Using Information Divergence Frontiers
Josip Djolonga, Mario Lucic, Marco Cuturi +3
Despite the tremendous progress in the estimation of generative models, the development of tools for diagnosing their failures and assessing their performance has advanced at a muc…
High-Fidelity Image Generation With Fewer Labels
Mario Lucic, Michael Tschannen, Marvin Ritter +3
Deep generative models are becoming a cornerstone of modern machine learning. Recent work on conditional generative adversarial networks has shown that learning complex, high-dimen…
Breaking the Softmax Bottleneck via Learnable Monotonic Pointwise Non-linearities
Octavian-Eugen Ganea, Sylvain Gelly, Gary Bécigneul +1
The Softmax function on top of a final linear layer is the de facto method to output probability distributions in neural networks. In many applications such as language models or t…
Parameter-Efficient Transfer Learning for NLP
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski +5
Fine-tuning large pre-trained models is an effective transfer mechanism in NLP. However, in the presence of many downstream tasks, fine-tuning is parameter inefficient: an entire n…
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
Francesco Locatello, Stefan Bauer, Mario Lucic +4
The key idea behind the unsupervised learning of disentangled representations is that real-world data is generated by a few explanatory factors of variation which can be recovered…
Episodic Curiosity through Reachability
Nikolay Savinov, Anton Raichuk, Raphaël Marinier +4
Rewards are sparse in the real world and most of today's reinforcement learning algorithms struggle with such sparsity. One solution to this problem is to allow the agent to create…
On Self Modulation for Generative Adversarial Networks
Ting Chen, Mario Lucic, Neil Houlsby +1
Training Generative Adversarial Networks (GANs) is notoriously challenging. We propose and study an architectural modification, self-modulation, which improves GAN performance acro…
Temporal Difference Learning with Neural Networks - Study of the Leakage Propagation Problem
Hugo Penedones, Damien Vincent, Hartmut Maennel +3
Temporal-Difference learning (TD) [Sutton, 1988] with function approximation can converge to solutions that are worse than those obtained by Monte-Carlo regression, even in the sim…
A Large-Scale Study on Regularization and Normalization in GANs
Karol Kurach, Mario Lucic, Xiaohua Zhai +2
Generative adversarial networks (GANs) are a class of deep generative models which aim to learn a target distribution in an unsupervised fashion. While they were successfully appli…
Competitive Training of Mixtures of Independent Deep Generative Models
Francesco Locatello, Damien Vincent, Ilya Tolstikhin +3
A common assumption in causal modeling posits that the data is generated by a set of independent mechanisms, and algorithms should aim to recover this structure. Standard unsupervi…
MemGEN: Memory is All You Need
Sylvain Gelly, Karol Kurach, Marcin Michalski +1
We propose a new learning paradigm called Deep Memory. It has the potential to completely revolutionize the Machine Learning field. Surprisingly, this paradigm has not been reinven…
Toward Optimal Run Racing: Application to Deep Learning Calibration
Olivier Bousquet, Sylvain Gelly, Karol Kurach +4
This paper aims at one-shot learning of deep neural nets, where a highly parallel setting is considered to address the algorithm calibration problem - selecting the best neural arc…
Critical Hyper-Parameters: No Random, No Cry
Olivier Bousquet, Sylvain Gelly, Karol Kurach +2
The selection of hyper-parameters is critical in Deep Learning. Because of the long training time of complex models and the availability of compute resources in the cloud, "one-sho…