4 papers
Spectral Analysis of Heavy-Ball Q-value Iteration
Donghwan Lee
We study the convergence and acceleration of Qvalue iteration (QVI) with heavy-ball momentum. Although acceleration of value iteration has been studied extensively, there has been…
Spectral Analysis of Dueling Q-Learning
Donghwan Lee
Q-learning is a fundamental algorithm in reinforcement learning (RL) for solving discounted Markov decision processes (MDPs) when the transition kernel is unknown. The deep Q-netwo…
Heavy-Ball Q-Learning with Residual Weighting Correction
Donghwan Lee
This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes convergence of its deterministic mean dynamics. It also identifies cond…
Geometrically Averaged Hard Target Updates for Linear Q-Learning
Donghwan Lee
Periodic hard target updates are among the most common stabilization devices in modern deep Q-learning. Recent studies suggest that target updates can improve stability in Q-learni…