7.8k citations · 21.1k across the 144 of their papers we have counts for
12 papers · 1 filter
Continual Unsupervised Representation Learning
Dushyant Rao, Francesco Visin, Andrei A. Rusu +3
Continual learning aims to improve the ability of modern learning systems to deal with non-stationary distributions, typically by attempting to learn a series of tasks sequentially…
Stabilizing Transformers for Reinforcement Learning
Emilio Parisotto, H. Francis Song, Jack W. Rae +10
Owing to their ability to both effectively integrate information over long time horizons and scale to massive amounts of data, self-attention architectures have recently shown brea…
Improving the Gating Mechanism of Recurrent Neural Networks
Albert Gu, Caglar Gulcehre, Tom Le Paine +2
Gating mechanisms are widely used in neural network models, where they allow gradients to backpropagate more easily through depth or time. However, their saturation property introd…
Meta-Learning with Warped Gradient Descent
Sebastian Flennerhag, Andrei A. Rusu, Razvan Pascanu +3
Learning an efficient update rule from data that promotes rapid learning of new tasks from the same distribution remains an open problem in meta-learning. Typically, previous works…
Task Agnostic Continual Learning via Meta Learning
Xu He, Jakub Sygnowski, Alexandre Galashov +3
While neural networks are powerful function approximators, they suffer from catastrophic forgetting when the data distribution is not stationary. One particular formalism that stud…
Information asymmetry in KL-regularized RL
Alexandre Galashov, Siddhant M. Jayakumar, Leonard Hasenclever +7
Many real world tasks exhibit rich structure that is repeated across different parts of the state space or in time. In this work we study the possibility of leveraging such repeate…
Meta-learning of Sequential Strategies
Pedro A. Ortega, Jane X. Wang, Mark Rowland +21
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. O…
Ray Interference: a Source of Plateaus in Deep Reinforcement Learning
Tom Schaul, Diana Borsa, Joseph Modayil +1
Rather than proposing a new method, this paper investigates an issue present in existing learning algorithms. We study the learning dynamics of reinforcement learning (RL), specifi…
A RAD approach to deep mixture models
Laurent Dinh, Jascha Sohl-Dickstein, Hugo Larochelle +1
Flow based models such as Real NVP are an extremely powerful approach to density estimation. However, existing flow based models are restricted to transforming continuous densities…
Exploiting Hierarchy for Learning and Transfer in KL-regularized RL
Dhruva Tirumala, Hyeonwoo Noh, Alexandre Galashov +6
As reinforcement learning agents are tasked with solving more challenging and diverse tasks, the ability to incorporate prior knowledge into the learning system and to exploit reus…
Distilling Policy Distillation
Wojciech Marian Czarnecki, Razvan Pascanu, Simon Osindero +3
The transfer of knowledge from one policy to another is an important tool in Deep Reinforcement Learning. This process, referred to as distillation, has been used to great success,…
Functional Regularisation for Continual Learning with Gaussian Processes
Michalis K. Titsias, Jonathan Schwarz, Alexander G. de G. Matthews +2
We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network. This method, referred…