265 citations · 1.6k across the 43 of their papers we have counts for
22 papers · 1 filter
Equivariant MuZero
Andreea Deac, Théophane Weber, George Papamakarios
Deep reinforcement learning repeatedly succeeds in closed, well-defined domains such as games (Chess, Go, StarCraft). The next frontier is real-world scenarios, where setups are nu…
Investigating the role of model-based learning in exploration and transfer
Jacob Walker, Eszter Vértes, Yazhe Li +4
State of the art reinforcement learning has enabled training agents on tasks of ever increasing complexity. However, the current paradigm tends to favor training agents from scratc…
Large-Scale Retrieval for Reinforcement Learning
Peter C. Humphreys, Arthur Guez, Olivier Tieleman +3
Effective decision making involves flexibly relating past experiences and relevant contextual information to a novel situation. In deep reinforcement learning (RL), the dominant pa…
Retrieval-Augmented Reinforcement Learning
Anirudh Goyal, Abram L. Friesen, Andrea Banino +13
Most deep reinforcement learning (RL) algorithms distill experience into parametric behavior policies or value functions via gradient updates. While effective, this approach has se…
Procedural Generalization by Planning with Self-Supervised World Models
Ankesh Anand, Jacob Walker, Yazhe Li +5
One of the key promises of model-based reinforcement learning is the ability to generalize using an internal model of the world to make predictions in novel environments and tasks.…
Muesli: Combining Improvements in Policy Optimization
Matteo Hessel, Ivo Danihelka, Fabio Viola +6
We propose a novel policy update that combines regularized policy optimization with model learning as an auxiliary loss. The update (henceforth Muesli) matches MuZero's state-of-th…
Synthetic Returns for Long-Term Credit Assignment
David Raposo, Sam Ritter, Adam Santoro +5
Since the earliest days of reinforcement learning, the workhorse method for assigning credit to actions over time has been temporal-difference (TD) learning, which propagates credi…
Neural Recursive Belief States in Multi-Agent Reinforcement Learning
Pol Moreno, Edward Hughes, Kevin R. McKee +2
In multi-agent reinforcement learning, the problem of learning to act is particularly difficult because the policies of co-players may be heavily conditioned on information only ob…
Counterfactual Credit Assignment in Model-Free Reinforcement Learning
Thomas Mesnard, Théophane Weber, Fabio Viola +11
Credit assignment in reinforcement learning is the problem of measuring an action's influence on future rewards. In particular, this requires separating skill from luck, i.e. disen…
Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokoban
Peter Karkus, Mehdi Mirza, Arthur Guez +5
Intelligent robots need to achieve abstract objectives using concrete, spatiotemporally complex sensory information and motor control. Tabula rasa deep reinforcement learning (RL)…
Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning
Giambattista Parascandolo, Lars Buesing, Josh Merel +6
Standard planners for sequential decision making (including Monte Carlo planning, tree search, dynamic programming, etc.) are constrained by an implicit sequential planning assumpt…
Causally Correct Partial Models for Reinforcement Learning
Danilo J. Rezende, Ivo Danihelka, George Papamakarios +11
In reinforcement learning, we can learn a model of future observations and rewards, and use it to plan the agent's next actions. However, jointly modeling future observations can b…
Value-driven Hindsight Modelling
Arthur Guez, Fabio Viola, Théophane Weber +5
Value estimation is a critical component of the reinforcement learning (RL) paradigm. The question of how to effectively learn value predictors from data is one of the major proble…
Combining Q-Learning and Search with Amortized Value Estimates
Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez +4
We introduce "Search with Amortized Value Estimates" (SAVE), an approach for combining model-free Q-learning with model-based Monte-Carlo Tree Search (MCTS). In SAVE, a learned pri…
Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions
Lars Buesing, Nicolas Heess, Theophane Weber
A plethora of problems in AI, engineering and the sciences are naturally formalized as inference in discrete probabilistic models. Exact inference is often prohibitively expensive,…
Credit Assignment Techniques in Stochastic Computation Graphs
Théophane Weber, Nicolas Heess, Lars Buesing +1
Stochastic computation graphs (SCGs) provide a formalism to represent structured optimization problems arising in artificial intelligence, including supervised, unsupervised, and r…
An investigation of model-free planning
Arthur Guez, Mehdi Mirza, Karol Gregor +10
The field of reinforcement learning (RL) is facing increasingly challenging domains with combinatorial complexity. For an RL agent to address these challenges, it is essential that…
Woulda, Coulda, Shoulda: Counterfactually-Guided Policy Search
Lars Buesing, Theophane Weber, Yori Zwols +4
Learning policies on data synthesized by models can in principle quench the thirst of reinforcement learning algorithms for large amounts of real experience, which is often costly…
Relational recurrent neural networks
Adam Santoro, Ryan Faulkner, David Raposo +7
Memory-based neural networks model temporal data by leveraging an ability to remember information for long periods. It is unclear, however, whether they also have an ability to per…
Learning and Querying Fast Generative Models for Reinforcement Learning
Lars Buesing, Theophane Weber, Sebastien Racaniere +8
A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed generative mo…
Imagination-Augmented Agents for Deep Reinforcement Learning
Théophane Weber, Sébastien Racanière, David P. Reichert +12
We introduce Imagination-Augmented Agents (I2As), a novel architecture for deep reinforcement learning combining model-free and model-based aspects. In contrast to most existing mo…
Gradient Estimation Using Stochastic Computation Graphs
John Schulman, Nicolas Heess, Theophane Weber +1
In a variety of problems originating in supervised, unsupervised, and reinforcement learning, the loss function is defined by an expectation over a collection of random variables,…