8 papers · 1 filter
Spectral Analysis of Dueling Q-Learning
Donghwan Lee
Q-learning is a fundamental algorithm in reinforcement learning (RL) for solving discounted Markov decision processes (MDPs) when the transition kernel is unknown. The deep Q-netwo…
Heavy-Ball Q-Learning with Residual Weighting Correction
Donghwan Lee
This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes convergence of its deterministic mean dynamics. It also identifies cond…
Geometrically Averaged Hard Target Updates for Linear Q-Learning
Donghwan Lee
Periodic hard target updates are among the most common stabilization devices in modern deep Q-learning. Recent studies suggest that target updates can improve stability in Q-learni…
Soft Deterministic Policy Gradient with Gaussian Smoothing
Hyunjun Na, Donghwan Lee
Deterministic policy gradient (DPG) is widely utilized for continuous control; however, it inherently relies on the differentiability of the critic with respect to the action durin…
A Switching System Theory of Q-Learning with Linear Function Approximation
Donghwan Lee, Han-Dong Lim
Q-learning is a fundamental algorithmic primitive in reinforcement learning. This paper develops a new framework for analyzing linear Q-learning from a switching linear system (SLS…
Switching Theory for Q-Learning
Donghwan Lee
Q-learning is a fundamental algorithmic primitive in reinforcement learning. This paper develops a new framework for analyzing constant step-size tabular Q-learning from a switchin…