11 papers
A Switching System Theory of Q-Learning with Linear Function Approximation
Donghwan Lee, Han-Dong Lim
Q-learning is a fundamental algorithmic primitive in reinforcement learning. This paper develops a new framework for analyzing linear Q-learning from a switching linear system (SLS…
Contraction-Aligned Analysis of Soft Bellman Residual Minimization with Weighted Lp-Norm for Markov Decision Problem
Hyukjun Yang, Han-Dong Lim, Donghwan Lee
The problem of solving Markov decision processes under function approximation remains a fundamental challenge, even under linear function approximation settings. A key difficulty a…
Learning the Model While Learning Q: Finite-Time Sample Complexity of Online SyncMBQ
Han-Dong Lim, HyeAnn Lee, Donghwan Lee
Reinforcement learning has witnessed significant advancements, particularly with the emergence of model-based approaches. Among these, -learning has proven to be a powerful algo…
Analysis of Off-Policy -Step TD-Learning with Linear Function Approximation
Han-Dong Lim, Donghwan Lee
This paper analyzes multi-step temporal difference (TD)-learning algorithms within the ``deadly triad'' scenario, characterized by linear function approximation, off-policy learnin…
Periodic Regularized Q-Learning
Hyukjun Yang, Han-Dong Lim, Donghwan Lee
In reinforcement learning (RL), Q-learning is a fundamental algorithm whose convergence is guaranteed in the tabular setting. However, this convergence guarantee does not hold unde…
A finite time analysis of distributed Q-learning
Han-Dong Lim, Donghwan Lee
Multi-agent reinforcement learning (MARL) has witnessed a remarkable surge in interest, fueled by the empirical success achieved in applications of single-agent reinforcement learn…