21 citations · 149 across the 34 of their papers we have counts for
37 papers
Investigating Action Encodings in Recurrent Neural Networks in Reinforcement Learning
Matthew Schlegel, Volodymyr Tkachuk, Adam White +1
Building and maintaining state to learn policies and value functions is critical for deploying reinforcement learning (RL) agents in the real world. Recurrent neural networks (RNNs…
Measuring and Mitigating Interference in Reinforcement Learning
Vincent Liu, Han Wang, Ruo Yu Tao +3
Catastrophic interference is common in many network-based learning systems, and many proposals exist for mitigating it. Before overcoming interference we must understand it better.…
Empirical Design in Reinforcement Learning
Andrew Patterson, Samuel Neumann, Martha White +1
Empirical design in reinforcement learning is no small task. Running good experiments requires attention to detail and at times significant computational resources. While compute r…
Loss of Plasticity in Continual Deep Reinforcement Learning
Zaheer Abbas, Rosie Zhao, Joseph Modayil +2
The ability to learn continually is essential in a complex and changing world. In this paper, we characterize the behavior of canonical value-based deep reinforcement learning (RL)…
The In-Sample Softmax for Offline Reinforcement Learning
Chenjun Xiao, Han Wang, Yangchen Pan +2
Reinforcement learning (RL) agents can leverage batches of previously collected data to extract a reasonable control policy. An emerging issue in this offline RL setting, however,…
Agent-State Construction with Auxiliary Inputs
Ruo Yu Tao, Adam White, Marlos C. Machado
In many, if not every realistic sequential decision-making task, the decision-making agent is not able to model the full complexity of the world. The environment is often much larg…
Auxiliary task discovery through generate-and-test
Banafsheh Rafiee, Sina Ghiassian, Jun Jin +3
In this paper, we explore an approach to auxiliary task discovery in reinforcement learning based on ideas from representation learning. Auxiliary tasks tend to improve data effici…
Goal-Space Planning with Subgoal Models
Chunlok Lo, Kevin Roice, Parham Mohammad Panahi +5
This paper investigates a new approach to model-based reinforcement learning using background planning: mixing (approximate) dynamic programming updates and model-free updates, sim…
No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RL
Han Wang, Archit Sakhadeo, Adam White +7
The performance of reinforcement learning (RL) agents is sensitive to the choice of hyperparameters. In real-world settings like robotics or industrial control systems, however, te…
What makes useful auxiliary tasks in reinforcement learning: investigating the effect of the target policy
Banafsheh Rafiee, Jun Jin, Jun Luo +1
Auxiliary tasks have been argued to be useful for representation learning in reinforcement learning. Although many auxiliary tasks have been empirically shown to be effective for a…
The Frost Hollow Experiments: Pavlovian Signalling as a Path to Coordination and Communication Between Agents
Patrick M. Pilarski, Andrew Butcher, Elnaz Davoodi +7
Learned communication between agents is a powerful tool when approaching decision-making problems that are hard to overcome by any single agent in isolation. However, continual coo…
Investigating the Properties of Neural Network Representations in Reinforcement Learning
Han Wang, Erfan Miahi, Martha White +5
In this paper we investigate the properties of representations learned by deep reinforcement learning systems. Much of the early work on representations for reinforcement learning…
Continual Auxiliary Task Learning
Matthew McLeod, Chunlok Lo, Matthew Schlegel +4
Learning auxiliary tasks, such as multiple predictions about the world, can provide many benefits to reinforcement learning systems. A variety of off-policy learning algorithms hav…
Reward-Respecting Subtasks for Model-Based Reinforcement Learning
Richard S. Sutton, Marlos C. Machado, G. Zacharias Holland +4
To achieve the ambitious goals of artificial intelligence, reinforcement learning must include planning with a model of the world that is abstract in state and time. Deep learning…
Pavlovian Signalling with General Value Functions in Agent-Agent Temporal Decision Making
Andrew Butcher, Michael Bradley Johanson, Elnaz Davoodi +6
In this paper, we contribute a multi-faceted study into Pavlovian signalling -- a process by which learned, temporally extended predictions made by one agent inform decision-making…
Assessing Human Interaction in Virtual Reality With Continually Learning Prediction Agents Based on Reinforcement Learning Algorithms: A Pilot Study
Dylan J. A. Brenneis, Adam S. Parker, Michael Bradley Johanson +7
Artificial intelligence systems increasingly involve continual learning to enable flexibility in general situations that are not encountered during system training. Human interacti…
Learning Expected Emphatic Traces for Deep RL
Ray Jiang, Shangtong Zhang, Veronica Chelu +2
Off-policy sampling and experience replay are key for improving sample efficiency and scaling model-free temporal difference learning methods. When combined with function approxima…
Emphatic Algorithms for Deep Reinforcement Learning
Ray Jiang, Tom Zahavy, Zhongwen Xu +4
Off-policy learning allows us to learn about possible policies of behavior from experience generated by a different behavior policy. Temporal difference (TD) learning algorithms ca…
A Generalized Projected Bellman Error for Off-policy Value Estimation in Reinforcement Learning
Andrew Patterson, Adam White, Martha White
Many reinforcement learning algorithms rely on value estimation, however, the most widely used algorithms -- namely temporal difference algorithms -- can diverge under both off-pol…
From Eye-blinks to State Construction: Diagnostic Benchmarks for Online Representation Learning
Banafsheh Rafiee, Zaheer Abbas, Sina Ghiassian +4
We present three new diagnostic prediction problems inspired by classical-conditioning experiments to facilitate research in online prediction learning. Experiments in classical co…
Towards a practical measure of interference for reinforcement learning
Vincent Liu, Adam White, Hengshuai Yao +1
Catastrophic interference is common in many network-based learning systems, and many proposals exist for mitigating it. But, before we overcome interference we must understand it b…
Gradient Temporal-Difference Learning with Regularized Corrections
Sina Ghiassian, Andrew Patterson, Shivam Garg +3
It is still common to use Q-learning and temporal difference (TD) learning-even though they have divergence issues and sound Gradient TD alternatives exist-because divergence seems…
Improving Performance in Reinforcement Learning by Breaking Generalization in Neural Networks
Sina Ghiassian, Banafsheh Rafiee, Yat Long Lo +1
Reinforcement learning systems require good representations to work well. For decades practical success in reinforcement learning was limited to small domains. Deep reinforcement l…
Meta-descent for Online, Continual Prediction
Andrew Jacobsen, Matthew Schlegel, Cameron Linke +3
This paper investigates different vector step-size adaptation approaches for non-stationary online, continual prediction problems. Vanilla stochastic gradient descent can be consid…