93 citations · 487 across the 42 of their papers we have counts for
47 papers
Mastering Atari 2600 Games with Discovered Options
Erik M. Lintunen, Marlos C. Machado
Temporal abstractions, often instantiated as options, have long been regarded as a mechanism for accelerating credit assignment, facilitating exploration, and enabling generalisati…
Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira +1
Existing approaches to offline goal-conditioned reinforcement learning (GCRL) struggle with long-horizon tasks. Discounting shrinks value differences between distant states until t…
Accelerating Q-learning through Efficient Value-Sharing across Actions
Prabhat Nagarajan, Brett Daley, Martha White +1
Action values are foundational to many control algorithms such as Q-learning. Therefore, efficient action-value learning is central to reinforcement learning (RL). However, learnin…
Laplacian Representations for Decision-Time Planning
Dikshant Shehmar, Matthew Schlegel, Matthew E. Taylor +1
Planning with a learned model remains a key challenge in model-based reinforcement learning (RL). In decision-time planning, state representations are critical as they must support…
DROGO: Default Representation Objective via Graph Optimization in Reinforcement Learning
Hon Tik Tse, Marlos C. Machado
In computational reinforcement learning, the default representation (DR) and its principal eigenvector have been shown to be effective for a wide variety of applications, including…
An Analysis of Action-Value Temporal-Difference Methods That Learn State Values
Brett Daley, Prabhat Nagarajan, Martha White +1
The hallmark feature of temporal-difference (TD) learning is bootstrapping: using value predictions to generate new value predictions. The vast majority of TD methods for control l…
Deep Reinforcement Learning with Gradient Eligibility Traces
Esraa Elelimy, Brett Daley, Andrew Patterson +3
Achieving fast and stable off-policy learning in deep reinforcement learning (RL) is challenging. Most existing methods rely on semi-gradient temporal-difference (TD) methods for t…
Deep Double Q-learning
Prabhat Nagarajan, Martha White, Marlos C. Machado
Double Q-learning is a classical control algorithm that mitigates the maximization bias of Q-learning. To do so, it explicitly trains two independent action-value functions and use…
Discovering Temporal Structure: An Overview of Hierarchical Reinforcement Learning
Martin Klissarov, Akhil Bagaria, Ziyan Luo +3
Developing agents capable of exploring, planning and learning in complex open-ended environments is a grand challenge in artificial intelligence (AI). Hierarchical reinforcement le…
The Cell Must Go On: Agar.io for Continual Reinforcement Learning
Mohamed A. Mohamed, Kateryna Nekhomiazh, Vedant Vyas +3
Continual reinforcement learning (RL) concerns agents that are expected to learn continually, rather than converge to a policy that is then fixed for evaluation. This setting is we…
Reward-Aware Proto-Representations in Reinforcement Learning
Hon Tik Tse, Siddarth Chandrasekar, Marlos C. Machado
In recent years, the successor representation (SR) has attracted increasing attention in reinforcement learning (RL), and it has been used to address some of its key challenges, su…
MaestroMotif: Skill Design from Artificial Intelligence Feedback
Martin Klissarov, Mikael Henaff, Roberta Raileanu +7
Describing skills in natural language has the potential to provide an accessible way to inject human knowledge about decision-making into an AI system. We present MaestroMotif, a m…
Plastic Learning with Deep Fourier Features
Alex Lewandowski, Dale Schuurmans, Marlos C. Machado
Deep neural networks can struggle to learn continually in the face of non-stationarity. This phenomenon is known as loss of plasticity. In this paper, we identify underlying princi…
Demystifying the Recency Heuristic in Temporal-Difference Learning
Brett Daley, Marlos C. Machado, Martha White
The recency heuristic in reinforcement learning is the assumption that stimuli that occurred closer in time to an acquired reward should be more heavily reinforced. The recency heu…
Learning Continually by Spectral Regularization
Alex Lewandowski, Michał Bortkiewicz, Saurabh Kumar +4
Loss of plasticity is a phenomenon where neural networks can become more difficult to train over the course of learning. Continual learning algorithms seek to mitigate this effect…
Averaging -step Returns Reduces Variance in Reinforcement Learning
Brett Daley, Martha White, Marlos C. Machado
Multistep returns, such as -step returns and -returns, are commonly used to improve the sample efficiency of reinforcement learning (RL) methods. The variance of the multiste…
GVFs in the Real World: Making Predictions Online for Water Treatment
Muhammad Kamran Janjua, Haseeb Shah, Martha White +3
In this paper we investigate the use of reinforcement-learning based prediction approaches for a real drinking-water treatment plant. Developing such a prediction system is a criti…
Harnessing Discrete Representations For Continual Reinforcement Learning
Edan Meyer, Adam White, Marlos C. Machado
Reinforcement learning (RL) agents make decisions using nothing but observations from the environment, and consequently, heavily rely on the representations of those observations.…
Directions of Curvature as an Explanation for Loss of Plasticity
Alex Lewandowski, Haruto Tanaka, Dale Schuurmans +1
Loss of plasticity is a phenomenon in which neural networks lose their ability to learn from new experience. Despite being empirically observed in several problem settings, little…
AGaLiTe: Approximate Gated Linear Transformers for Online Reinforcement Learning
Subhojeet Pramanik, Esraa Elelimy, Marlos C. Machado +1
In this paper we investigate transformer architectures designed for partially observable online reinforcement learning. The self-attention mechanism in the transformer architecture…
Proper Laplacian Representation Learning
Diego Gomez, Michael Bowling, Marlos C. Machado
The ability to learn good representations of states is essential for solving large reinforcement learning problems, where exploration, generalization, and transfer are particularly…
Loss of Plasticity in Continual Deep Reinforcement Learning
Zaheer Abbas, Rosie Zhao, Joseph Modayil +2
The ability to learn continually is essential in a complex and changing world. In this paper, we characterize the behavior of canonical value-based deep reinforcement learning (RL)…
Trajectory-Aware Eligibility Traces for Off-Policy Reinforcement Learning
Brett Daley, Martha White, Christopher Amato +1
Off-policy learning from multistep returns is crucial for sample-efficient reinforcement learning, but counteracting off-policy bias without exacerbating variance is challenging. C…
Deep Laplacian-based Options for Temporally-Extended Exploration
Martin Klissarov, Marlos C. Machado
Selecting exploratory actions that generate a rich stream of experience for better learning is a fundamental challenge in reinforcement learning (RL). An approach to tackle this pr…