3.8k citations · 6.5k across the 56 of their papers we have counts for
12 papers · 1 filter
Continual Unsupervised Representation Learning
Dushyant Rao, Francesco Visin, Andrei A. Rusu +3
Continual learning aims to improve the ability of modern learning systems to deal with non-stationary distributions, typically by attempting to learn a series of tasks sequentially…
Stabilizing Transformers for Reinforcement Learning
Emilio Parisotto, H. Francis Song, Jack W. Rae +10
Owing to their ability to both effectively integrate information over long time horizons and scale to massive amounts of data, self-attention architectures have recently shown brea…
Improving the Gating Mechanism of Recurrent Neural Networks
Albert Gu, Caglar Gulcehre, Tom Le Paine +2
Gating mechanisms are widely used in neural network models, where they allow gradients to backpropagate more easily through depth or time. However, their saturation property introd…
Meta-Learning with Warped Gradient Descent
Sebastian Flennerhag, Andrei A. Rusu, Razvan Pascanu +3
Learning an efficient update rule from data that promotes rapid learning of new tasks from the same distribution remains an open problem in meta-learning. Typically, previous works…
Task Agnostic Continual Learning via Meta Learning
Xu He, Jakub Sygnowski, Alexandre Galashov +3
While neural networks are powerful function approximators, they suffer from catastrophic forgetting when the data distribution is not stationary. One particular formalism that stud…
Information asymmetry in KL-regularized RL
Alexandre Galashov, Siddhant M. Jayakumar, Leonard Hasenclever +7
Many real world tasks exhibit rich structure that is repeated across different parts of the state space or in time. In this work we study the possibility of leveraging such repeate…