6 papers
Gated Q-learning: Add Off-Policy Bias to Taste
Brett Daley
Multistep credit assignment is critical for sample-efficient reinforcement learning, yet managing off-policy bias in Q-learning remains a fundamental challenge. For 30 years, pract…
Accelerating Q-learning through Efficient Value-Sharing across Actions
Prabhat Nagarajan, Brett Daley, Martha White +1
Action values are foundational to many control algorithms such as Q-learning. Therefore, efficient action-value learning is central to reinforcement learning (RL). However, learnin…
Averaging -step Returns Reduces Variance in Reinforcement Learning
Brett Daley, Martha White, Marlos C. Machado
Multistep returns, such as -step returns and -returns, are commonly used to improve the sample efficiency of reinforcement learning (RL) methods. The variance of the multist…
Deep Reinforcement Learning with Gradient Eligibility Traces
Esraa Elelimy, Brett Daley, Andrew Patterson +3
Achieving fast and stable off-policy learning in deep reinforcement learning (RL) is challenging. Most existing methods rely on semi-gradient temporal-difference (TD) methods for t…
An Analysis of Action-Value Temporal-Difference Methods That Learn State Values
Brett Daley, Prabhat Nagarajan, Martha White +1
The hallmark feature of temporal-difference (TD) learning is bootstrapping: using value predictions to generate new value predictions. The vast majority of TD methods for control l…
Demystifying the Recency Heuristic in Temporal-Difference Learning
Brett Daley, Marlos C. Machado, Martha White
The recency heuristic in reinforcement learning is the assumption that stimuli that occurred closer in time to an acquired reward should be more heavily reinforced. The recency heu…