424 citations · 1.3k across the 18 of their papers we have counts for
21 papers
Deep Reinforcement Learning with Plasticity Injection
Evgenii Nikishin, Junhyuk Oh, Georg Ostrovski +4
A growing body of evidence suggests that neural networks employed in deep reinforcement learning (RL) gradually lose their plasticity, the ability to learn from new data; however,…
An Analysis of Quantile Temporal-Difference Learning
Mark Rowland, Rémi Munos, Mohammad Gheshlaghi Azar +6
We analyse quantile temporal-difference learning (QTD), a distributional reinforcement learning algorithm that has proven to be a key component in several successful large-scale ap…
An Empirical Study of Implicit Regularization in Deep Offline RL
Caglar Gulcehre, Srivatsan Srinivasan, Jakub Sygnowski +5
Deep neural networks are the most commonly used function approximators in offline reinforcement learning. Prior works have shown that neural nets trained with TD-learning and gradi…
The Phenomenon of Policy Churn
Tom Schaul, André Barreto, John Quan +1
We identify and study the phenomenon of policy churn, that is, the rapid change of the greedy policy in value-based reinforcement learning. Policy churn operates at a surprisingly…
The Difficulty of Passive Learning in Deep Reinforcement Learning
Georg Ostrovski, Pablo Samuel Castro, Will Dabney
Learning to act from observational data without active environmental interaction is a well-known challenge in Reinforcement Learning (RL). Recent approaches involve constraints on…
When should agents explore?
Miruna Pîslar, David Szepesvari, Georg Ostrovski +2
Exploration remains a central challenge for reinforcement learning (RL). Virtually all existing methods share the feature of a monolithic behaviour policy that changes only gradual…
Return-based Scaling: Yet Another Normalisation Trick for Deep RL
Tom Schaul, Georg Ostrovski, Iurii Kemaev +1
Scaling issues are mundane yet irritating for practitioners of reinforcement learning. Error scales vary across domains, tasks, and stages of learning; sometimes by many orders of…
On The Effect of Auxiliary Tasks on Representation Dynamics
Clare Lyle, Mark Rowland, Georg Ostrovski +1
While auxiliary tasks play a key role in shaping the representations learnt by reinforcement learning agents, much is still unknown about the mechanisms through which this is achie…
Temporally-Extended ε-Greedy Exploration
Will Dabney, Georg Ostrovski, André Barreto
Recent work on exploration in reinforcement learning (RL) has led to a series of increasingly complex solutions to the problem. This increase in complexity often comes at the expen…
Adapting Behaviour for Learning Progress
Tom Schaul, Diana Borsa, David Ding +4
Determining what experience to generate to best facilitate learning (i.e. exploration) is one of the distinguishing features and open challenges in reinforcement learning. The adve…
Implicit Quantile Networks for Distributional Reinforcement Learning
Will Dabney, Georg Ostrovski, David Silver +1
In this work, we build on recent advances in distributional reinforcement learning to give a generally applicable, flexible, and state-of-the-art distributional variant of DQN. We…
Autoregressive Quantile Networks for Generative Modeling
Georg Ostrovski, Will Dabney, Rémi Munos
We introduce autoregressive implicit quantile networks (AIQN), a fundamentally different approach to generative modeling than those commonly used, that implicitly captures the dist…
Symmetric Decomposition of Asymmetric Games
Karl Tuyls, Julien Perolat, Marc Lanctot +6
We introduce new theoretical insights into two-population asymmetric games allowing for an elegant symmetric decomposition into two single population symmetric games. Specifically,…
Rainbow: Combining Improvements in Deep Reinforcement Learning
Matteo Hessel, Joseph Modayil, Hado van Hasselt +7
The deep reinforcement learning community has made several independent improvements to the DQN algorithm. However, it is unclear which of these extensions are complementary and can…
Count-Based Exploration with Neural Density Models
Georg Ostrovski, Marc G. Bellemare, Aaron van den Oord +1
Bellemare et al. (2016) introduced the notion of a pseudo-count, derived from a density model, to generalize count-based exploration to non-tabular reinforcement learning. This pse…
Unifying Count-Based Exploration and Intrinsic Motivation
Marc G. Bellemare, Sriram Srinivasan, Georg Ostrovski +3
We consider an agent's uncertainty about its environment and the problem of generalizing this uncertainty across observations. Specifically, we focus on the problem of exploration…
Increasing the Action Gap: New Operators for Reinforcement Learning
Marc G. Bellemare, Georg Ostrovski, Arthur Guez +2
This paper introduces new optimality-preserving operators on Q-functions. We first describe an operator for tabular representations, the consistent Bellman operator, which incorpor…
Payoff Performance of Fictitious Play
Georg Ostrovski, Sebastian van Strien
We investigate how well continuous-time fictitious play in two-player games performs in terms of average payoff, particularly compared to Nash equilibrium payoff. We show that in m…
Dynamics of a Continuous Piecewise Affine Map of the Square
Georg Ostrovski
We present a one-parameter family of continuous, piecewise affine, area preserving maps of the square, which are inspired by a dynamical system in game theory. Interested in the co…
Fixed Point Theorem for Non-Self Maps of Regions in the Plane
Georg Ostrovski
Let X and Y be compact, simply connected and locally connected subsets of R^2, and let f : X -> Y be a homeomorphism isotopic to the identity on X. Generalizing Brouwer's plane tra…
Piecewise Linear Hamiltonian Flows Associated to Zero-Sum Games: Transition Combinatorics and Questions on Ergodicity
Georg Ostrovski, Sebastian van Strien
In this paper we consider a class of piecewise affine Hamiltonian vector fields whose orbits are piecewise straight lines. We give a first classification result of such systems and…