39 citations · 198 across the 21 of their papers we have counts for
23 papers
Wasserstein Policy Optimization
David Pfau, Ian Davies, Diana Borsa +3
We introduce Wasserstein Policy Optimization (WPO), an actor-critic algorithm for reinforcement learning in continuous action spaces. WPO can be derived as an approximation to Wass…
Optimizing Return Distributions with Distributional Dynamic Programming
Bernardo Ávila Pires, Mark Rowland, Diana Borsa +6
We introduce distributional dynamic programming (DP) methods for optimizing statistical functionals of the return distribution, with standard reinforcement learning as a special ca…
A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning
Khimya Khetarpal, Zhaohan Daniel Guo, Bernardo Avila Pires +7
Learning a good representation is a crucial challenge for Reinforcement Learning (RL) agents. Self-predictive learning provides means to jointly learn a latent representation and d…
A State Representation for Diminishing Rewards
Ted Moskovitz, Samo Hromadka, Ahmed Touati +2
A common setting in multitask reinforcement learning (RL) demands that an agent rapidly adapt to various stationary reward functions randomly sampled from a fixed distribution. In…
Representations and Exploration for Deep Reinforcement Learning using Singular Value Decomposition
Yash Chandak, Shantanu Thakoor, Zhaohan Daniel Guo +4
Representation learning and exploration are among the key challenges for any deep reinforcement learning agent. In this work, we provide a singular value decomposition based method…
Generalised Policy Improvement with Geometric Policy Composition
Shantanu Thakoor, Mark Rowland, Diana Borsa +3
We introduce a method for policy improvement that interpolates between the greedy approach of value-based reinforcement learning (RL) and the full planning approach typical of mode…
Selective Credit Assignment
Veronica Chelu, Diana Borsa, Doina Precup +1
Efficient credit assignment is essential for reinforcement learning algorithms in both prediction and control settings. We describe a unified view on temporal-difference algorithms…
Model-Value Inconsistency as a Signal for Epistemic Uncertainty
Angelos Filos, Eszter Vértes, Zita Marinho +7
Using a model of the environment and a value function, an agent can construct many estimates of a state's value, by unrolling the model for different lengths and bootstrapping with…
When should agents explore?
Miruna Pîslar, David Szepesvari, Georg Ostrovski +2
Exploration remains a central challenge for reinforcement learning (RL). Virtually all existing methods share the feature of a monolithic behaviour policy that changes only gradual…
The Option Keyboard: Combining Skills in Reinforcement Learning
André Barreto, Diana Borsa, Shaobo Hou +8
The ability to combine known skills to create new ones may be crucial in the solution of complex reinforcement learning problems that unfold over extended periods. We argue that a…
Return-based Scaling: Yet Another Normalisation Trick for Deep RL
Tom Schaul, Georg Ostrovski, Iurii Kemaev +1
Scaling issues are mundane yet irritating for practitioners of reinforcement learning. Error scales vary across domains, tasks, and stages of learning; sometimes by many orders of…
Temporal Difference Uncertainties as a Signal for Exploration
Sebastian Flennerhag, Jane X. Wang, Pablo Sprechmann +7
An effective approach to exploration in reinforcement learning is to rely on an agent's uncertainty over the optimal policy, which can yield near-optimal exploration strategies in…
Expected Eligibility Traces
Hado van Hasselt, Sephora Madjiheurem, Matteo Hessel +3
The question of how to determine which states and actions are responsible for a certain outcome is known as the credit assignment problem and remains a central research question in…
Adapting Behaviour for Learning Progress
Tom Schaul, Diana Borsa, David Ding +4
Determining what experience to generate to best facilitate learning (i.e. exploration) is one of the distinguishing features and open challenges in reinforcement learning. The adve…
Conditional Importance Sampling for Off-Policy Learning
Mark Rowland, Anna Harutyunyan, Hado van Hasselt +4
The principal contribution of this paper is a conceptual framework for off-policy reinforcement learning, based on conditional expectations of importance sampling ratios. This fram…
General non-linear Bellman equations
Hado van Hasselt, John Quan, Matteo Hessel +3
We consider a general class of non-linear Bellman equations. These open up a design space of algorithms that have interesting properties, which has two potential advantages. First,…
Ray Interference: a Source of Plateaus in Deep Reinforcement Learning
Tom Schaul, Diana Borsa, Joseph Modayil +1
Rather than proposing a new method, this paper investigates an issue present in existing learning algorithms. We study the learning dynamics of reinforcement learning (RL), specifi…
The Termination Critic
Anna Harutyunyan, Will Dabney, Diana Borsa +3
In this work, we consider the problem of autonomously discovering behavioral abstractions, or options, for reinforcement learning agents. We propose an algorithm that focuses on th…
Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement
André Barreto, Diana Borsa, John Quan +6
The ability to transfer skills across tasks has the potential to scale up reinforcement learning (RL) agents to environments currently out of reach. Recently, a framework based on…
Universal Successor Features Approximators
Diana Borsa, André Barreto, John Quan +5
The ability of a reinforcement learning (RL) agent to learn about many reward functions at the same time has many potential benefits, such as the decomposition of complex tasks int…
Observational Learning by Reinforcement Learning
Diana Borsa, Bilal Piot, Rémi Munos +1
Observational learning is a type of learning that occurs as a function of observing, retaining and possibly replicating or imitating the behaviour of another agent. It is a core me…
Learning Shared Representations in Multi-task Reinforcement Learning
Diana Borsa, Thore Graepel, John Shawe-Taylor
We investigate a paradigm in multi-task reinforcement learning (MT-RL) in which an agent is placed in an environment and needs to learn to perform a series of tasks, within this sp…
The Wreath Process: A totally generative model of geometric shape based on nested symmetries
Diana Borsa, Thore Graepel, Andrew Gordon
We consider the problem of modelling noisy but highly symmetric shapes that can be viewed as hierarchies of whole-part relationships in which higher level objects are composed of t…