445 citations · 1.8k across the 17 of their papers we have counts for
19 papers
Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design
Matthew Thomas Jackson, Minqi Jiang, Jack Parker-Holder +5
The past decade has seen vast progress in deep reinforcement learning (RL) on the back of algorithms manually designed by human researchers. Recently, it has been shown that it is…
An Investigation of the Bias-Variance Tradeoff in Meta-Gradients
Risto Vuorio, Jacob Beck, Shimon Whiteson +2
Meta-gradients provide a general approach for optimizing the meta-parameters of reinforcement learning (RL) algorithms. Estimation of meta-gradients is central to the performance o…
Model-Value Inconsistency as a Signal for Epistemic Uncertainty
Angelos Filos, Eszter Vértes, Zita Marinho +7
Using a model of the environment and a value function, an agent can construct many estimates of a state's value, by unrolling the model for different lengths and bootstrapping with…
Self-Consistent Models and Values
Gregory Farquhar, Kate Baumli, Zita Marinho +4
Learned models of the environment provide reinforcement learning (RL) agents with flexible ways of making predictions about the environment. In particular, models enable planning,…
Proper Value Equivalence
Christopher Grimm, André Barreto, Gregory Farquhar +2
One of the main challenges in model-based reinforcement learning (RL) is to decide which aspects of the environment should be modeled. The value-equivalence (VE) principle proposes…
PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning
Angelos Filos, Clare Lyle, Yarin Gal +3
We study reinforcement learning (RL) with no-reward demonstrations, a setting in which an RL agent has access to additional data from the interaction of other agents with the same…
Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
Tabish Rashid, Gregory Farquhar, Bei Peng +1
QMIX is a popular -learning algorithm for cooperative MARL in the centralised training and decentralised execution paradigm. In order to enable easy decentralisation, QMIX restr…
Transient Non-Stationarity and Generalisation in Deep Reinforcement Learning
Maximilian Igl, Gregory Farquhar, Jelena Luketina +2
Non-stationarity can arise in Reinforcement Learning (RL) even in stationary environments. For example, most RL algorithms collect new data throughout training, using a non-station…
Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
Tabish Rashid, Mikayel Samvelyan, Christian Schroeder de Witt +3
In many real-world settings, a team of agents must coordinate its behaviour while acting in a decentralised fashion. At the same time, it is often possible to train the agents in a…
Loaded DiCE: Trading off Bias and Variance in Any-Order Score Function Estimators for Reinforcement Learning
Gregory Farquhar, Shimon Whiteson, Jakob Foerster
Gradient-based methods for optimisation of objectives in stochastic settings with unknown or intractable dynamics require estimators of derivatives. We derive an objective that, un…
Growing Action Spaces
Gregory Farquhar, Laura Gustafson, Zeming Lin +3
In complex tasks, such as those with large combinatorial action spaces, random exploration may be too inefficient to achieve meaningful learning progress. In this work, we use a cu…
A Survey of Reinforcement Learning Informed by Natural Language
Jelena Luketina, Nantas Nardelli, Gregory Farquhar +5
To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the compositional, relational, and hierarchical structure of the world, and learn to transfer it…
The StarCraft Multi-Agent Challenge
Mikayel Samvelyan, Tabish Rashid, Christian Schroeder de Witt +7
In the last few years, deep multi-agent reinforcement learning (RL) has become a highly active area of research. A particularly challenging class of problems in this area is partia…
Multi-Agent Common Knowledge Reinforcement Learning
Christian A. Schroeder de Witt, Jakob N. Foerster, Gregory Farquhar +3
Cooperative multi-agent reinforcement learning often requires decentralised policies, which severely limit the agents' ability to coordinate their behaviour. In this paper, we show…
QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
Tabish Rashid, Mikayel Samvelyan, Christian Schroeder de Witt +3
In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a c…
DiCE: The Infinitely Differentiable Monte-Carlo Estimator
Jakob Foerster, Gregory Farquhar, Maruan Al-Shedivat +3
The score function estimator is widely used for estimating gradients of stochastic objectives in stochastic computation graphs (SCG), eg, in reinforcement learning and meta-learnin…
TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning
Gregory Farquhar, Tim Rocktäschel, Maximilian Igl +1
Combining deep model-free reinforcement learning with on-line planning is a promising approach to building on the successes of deep RL. On-line planning with look-ahead trees has p…
Counterfactual Multi-Agent Policy Gradients
Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras +2
Cooperative multi-agent systems can be naturally used to model many real world problems, such as network packet routing and the coordination of autonomous vehicles. There is a grea…
Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
Jakob Foerster, Nantas Nardelli, Gregory Farquhar +4
Many real-world problems, such as network packet routing and urban traffic control, are naturally modeled as multi-agent reinforcement learning (RL) problems. However, existing mul…