39 citations · 198 across the 21 of their papers we have counts for
17 papers · 1 filter
Wasserstein Policy Optimization
David Pfau, Ian Davies, Diana Borsa +3
We introduce Wasserstein Policy Optimization (WPO), an actor-critic algorithm for reinforcement learning in continuous action spaces. WPO can be derived as an approximation to Wass…
Optimizing Return Distributions with Distributional Dynamic Programming
Bernardo Ávila Pires, Mark Rowland, Diana Borsa +6
We introduce distributional dynamic programming (DP) methods for optimizing statistical functionals of the return distribution, with standard reinforcement learning as a special ca…
A Unifying Framework for Action-Conditional Self-Predictive Reinforcement Learning
Khimya Khetarpal, Zhaohan Daniel Guo, Bernardo Avila Pires +7
Learning a good representation is a crucial challenge for Reinforcement Learning (RL) agents. Self-predictive learning provides means to jointly learn a latent representation and d…
A State Representation for Diminishing Rewards
Ted Moskovitz, Samo Hromadka, Ahmed Touati +2
A common setting in multitask reinforcement learning (RL) demands that an agent rapidly adapt to various stationary reward functions randomly sampled from a fixed distribution. In…
Representations and Exploration for Deep Reinforcement Learning using Singular Value Decomposition
Yash Chandak, Shantanu Thakoor, Zhaohan Daniel Guo +4
Representation learning and exploration are among the key challenges for any deep reinforcement learning agent. In this work, we provide a singular value decomposition based method…
Selective Credit Assignment
Veronica Chelu, Diana Borsa, Doina Precup +1
Efficient credit assignment is essential for reinforcement learning algorithms in both prediction and control settings. We describe a unified view on temporal-difference algorithms…
Model-Value Inconsistency as a Signal for Epistemic Uncertainty
Angelos Filos, Eszter Vértes, Zita Marinho +7
Using a model of the environment and a value function, an agent can construct many estimates of a state's value, by unrolling the model for different lengths and bootstrapping with…
When should agents explore?
Miruna Pîslar, David Szepesvari, Georg Ostrovski +2
Exploration remains a central challenge for reinforcement learning (RL). Virtually all existing methods share the feature of a monolithic behaviour policy that changes only gradual…
Return-based Scaling: Yet Another Normalisation Trick for Deep RL
Tom Schaul, Georg Ostrovski, Iurii Kemaev +1
Scaling issues are mundane yet irritating for practitioners of reinforcement learning. Error scales vary across domains, tasks, and stages of learning; sometimes by many orders of…
Expected Eligibility Traces
Hado van Hasselt, Sephora Madjiheurem, Matteo Hessel +3
The question of how to determine which states and actions are responsible for a certain outcome is known as the credit assignment problem and remains a central research question in…
Adapting Behaviour for Learning Progress
Tom Schaul, Diana Borsa, David Ding +4
Determining what experience to generate to best facilitate learning (i.e. exploration) is one of the distinguishing features and open challenges in reinforcement learning. The adve…
Conditional Importance Sampling for Off-Policy Learning
Mark Rowland, Anna Harutyunyan, Hado van Hasselt +4
The principal contribution of this paper is a conceptual framework for off-policy reinforcement learning, based on conditional expectations of importance sampling ratios. This fram…
General non-linear Bellman equations
Hado van Hasselt, John Quan, Matteo Hessel +3
We consider a general class of non-linear Bellman equations. These open up a design space of algorithms that have interesting properties, which has two potential advantages. First,…
Ray Interference: a Source of Plateaus in Deep Reinforcement Learning
Tom Schaul, Diana Borsa, Joseph Modayil +1
Rather than proposing a new method, this paper investigates an issue present in existing learning algorithms. We study the learning dynamics of reinforcement learning (RL), specifi…
Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvement
André Barreto, Diana Borsa, John Quan +6
The ability to transfer skills across tasks has the potential to scale up reinforcement learning (RL) agents to environments currently out of reach. Recently, a framework based on…
Universal Successor Features Approximators
Diana Borsa, André Barreto, John Quan +5
The ability of a reinforcement learning (RL) agent to learn about many reward functions at the same time has many potential benefits, such as the decomposition of complex tasks int…
Observational Learning by Reinforcement Learning
Diana Borsa, Bilal Piot, Rémi Munos +1
Observational learning is a type of learning that occurs as a function of observing, retaining and possibly replicating or imitating the behaviour of another agent. It is a core me…