7 papers · 1 filter
Accelerating Q-learning through Efficient Value-Sharing across Actions
Prabhat Nagarajan, Brett Daley, Martha White +1
Action values are foundational to many control algorithms such as Q-learning. Therefore, efficient action-value learning is central to reinforcement learning (RL). However, learnin…
An Analysis of Action-Value Temporal-Difference Methods That Learn State Values
Brett Daley, Prabhat Nagarajan, Martha White +1
The hallmark feature of temporal-difference (TD) learning is bootstrapping: using value predictions to generate new value predictions. The vast majority of TD methods for control l…
Deep Reinforcement Learning with Gradient Eligibility Traces
Esraa Elelimy, Brett Daley, Andrew Patterson +3
Achieving fast and stable off-policy learning in deep reinforcement learning (RL) is challenging. Most existing methods rely on semi-gradient temporal-difference (TD) methods for t…
Deep Double Q-learning
Prabhat Nagarajan, Martha White, Marlos C. Machado
Double Q-learning is a classical control algorithm that mitigates the maximization bias of Q-learning. To do so, it explicitly trains two independent action-value functions and use…
Demystifying the Recency Heuristic in Temporal-Difference Learning
Brett Daley, Marlos C. Machado, Martha White
The recency heuristic in reinforcement learning is the assumption that stimuli that occurred closer in time to an acquired reward should be more heavily reinforced. The recency heu…
Averaging -step Returns Reduces Variance in Reinforcement Learning
Brett Daley, Martha White, Marlos C. Machado
Multistep returns, such as -step returns and -returns, are commonly used to improve the sample efficiency of reinforcement learning (RL) methods. The variance of the multiste…