1.8k citations · 4.8k across the 37 of their papers we have counts for
30 papers · 1 filter
Normalization and effective learning rates in reinforcement learning
Clare Lyle, Zeyu Zheng, Khimya Khetarpal +4
Normalization layers have recently experienced a renaissance in the deep reinforcement learning and continual learning literature, with several works highlighting diverse benefits…
Disentangling the Causes of Plasticity Loss in Neural Networks
Clare Lyle, Zeyu Zheng, Khimya Khetarpal +4
Underpinning the past decades of work on the design, initialization, and optimization of neural networks is a seemingly innocuous assumption: that the network is trained on a \text…
A Definition of Continual Reinforcement Learning
David Abel, André Barreto, Benjamin Van Roy +3
In a standard view of the reinforcement learning problem, an agent's goal is to efficiently identify a policy that maximizes long-term reward. However, this perspective is based on…
Podracer architectures for scalable Reinforcement Learning
Matteo Hessel, Manuel Kroiss, Aidan Clark +5
Supporting state-of-the-art AI research requires balancing rapid prototyping, ease of use, and quick iteration, with the ability to deploy experiments at a scale traditionally asso…
Muesli: Combining Improvements in Policy Optimization
Matteo Hessel, Ivo Danihelka, Fabio Viola +6
We propose a novel policy update that combines regularized policy optimization with model learning as an auxiliary loss. The update (henceforth Muesli) matches MuZero's state-of-th…
Synthetic Returns for Long-Term Credit Assignment
David Raposo, Sam Ritter, Adam Santoro +5
Since the earliest days of reinforcement learning, the workhorse method for assigning credit to actions over time has been temporal-difference (TD) learning, which propagates credi…
Discovery of Options via Meta-Learned Subgoals
Vivek Veeriah, Tom Zahavy, Matteo Hessel +6
Temporal abstractions in the form of options have been shown to help reinforcement learning (RL) agents learn faster. However, despite prior work on this topic, the problem of disc…
Forethought and Hindsight in Credit Assignment
Veronica Chelu, Doina Precup, Hado van Hasselt
We address the problem of credit assignment in reinforcement learning and explore fundamental questions regarding the way in which an agent can best use additional computation to p…
Meta-Gradient Reinforcement Learning with an Objective Discovered Online
Zhongwen Xu, Hado van Hasselt, Matteo Hessel +3
Deep reinforcement learning includes a broad family of algorithms that parameterise an internal representation, such as a value function or policy, by a deep neural network. Each a…
Discovering Reinforcement Learning Algorithms
Junhyuk Oh, Matteo Hessel, Wojciech M. Czarnecki +4
Reinforcement learning (RL) algorithms update an agent's parameters according to one of several possible rules, discovered manually through years of research. Automating the discov…
Expected Eligibility Traces
Hado van Hasselt, Sephora Madjiheurem, Matteo Hessel +3
The question of how to determine which states and actions are responsible for a certain outcome is known as the credit assignment problem and remains a central research question in…
Hindsight Credit Assignment
Anna Harutyunyan, Will Dabney, Thomas Mesnard +8
We consider the problem of efficient credit assignment in reinforcement learning. In order to efficiently and meaningfully utilize new data, we propose to explicitly assign credit…
Conditional Importance Sampling for Off-Policy Learning
Mark Rowland, Anna Harutyunyan, Hado van Hasselt +4
The principal contribution of this paper is a conceptual framework for off-policy reinforcement learning, based on conditional expectations of importance sampling ratios. This fram…
Behaviour Suite for Reinforcement Learning
Ian Osband, Yotam Doron, Matteo Hessel +11
This paper introduces the Behaviour Suite for Reinforcement Learning, or bsuite for short. bsuite is a collection of carefully-designed experiments that investigate core capabiliti…
General non-linear Bellman equations
Hado van Hasselt, John Quan, Matteo Hessel +3
We consider a general class of non-linear Bellman equations. These open up a design space of algorithms that have interesting properties, which has two potential advantages. First,…
On Inductive Biases in Deep Reinforcement Learning
Matteo Hessel, Hado van Hasselt, Joseph Modayil +1
Many deep reinforcement learning algorithms contain inductive biases that sculpt the agent's objective and its interface to the environment. These inductive biases can take many fo…
When to use parametric models in reinforcement learning?
Hado van Hasselt, Matteo Hessel, John Aslanides
We examine the question of when and how parametric models are most useful in reinforcement learning. In particular, we look at commonalities and differences between parametric mode…
Meta-learning of Sequential Strategies
Pedro A. Ortega, Jane X. Wang, Mark Rowland +21
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. O…
Universal Successor Features Approximators
Diana Borsa, André Barreto, John Quan +5
The ability of a reinforcement learning (RL) agent to learn about many reward functions at the same time has many potential benefits, such as the decomposition of complex tasks int…
Multi-task Deep Reinforcement Learning with PopArt
Matteo Hessel, Hubert Soyer, Lasse Espeholt +3
The reinforcement learning community has made great strides in designing algorithms capable of exceeding human performance on specific tasks. These algorithms are mostly trained on…
Observe and Look Further: Achieving Consistent Performance on Atari
Tobias Pohlen, Bilal Piot, Todd Hester +10
Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks…
Meta-Gradient Reinforcement Learning
Zhongwen Xu, Hado van Hasselt, David Silver
The goal of reinforcement learning algorithms is to estimate and/or optimise the value function. However, unlike supervised learning, no teacher or oracle is available to provide t…
Distributed Prioritized Experience Replay
Dan Horgan, John Quan, David Budden +4
We propose a distributed architecture for deep reinforcement learning at scale, that enables agents to learn effectively from orders of magnitude more data than previously possible…
Unicorn: Continual Learning with a Universal, Off-policy Agent
Daniel J. Mankowitz, Augustin Žídek, André Barreto +7
Some real-world domains are best characterized as a single task, but for others this perspective is limiting. Instead, some tasks continually grow in complexity, in tandem with the…