4 papers
Contraction-Aligned Analysis of Soft Bellman Residual Minimization with Weighted Lp-Norm for Markov Decision Problem
Hyukjun Yang, Han-Dong Lim, Donghwan Lee
The problem of solving Markov decision processes under function approximation remains a fundamental challenge, even under linear function approximation settings. A key difficulty a…
Periodic Regularized Q-Learning
Hyukjun Yang, Han-Dong Lim, Donghwan Lee
In reinforcement learning (RL), Q-learning is a fundamental algorithm whose convergence is guaranteed in the tabular setting. However, this convergence guarantee does not hold unde…
Understanding the theoretical properties of projected Bellman equation, linear Q-learning, and approximate value iteration
Han-Dong Lim, Donghwan Lee
In this paper, we study the theoretical properties of the projected Bellman equation (PBE) and two algorithms to solve this equation: linear Q-learning and approximate value iterat…
Analysis of Off-Policy -Step TD-Learning with Linear Function Approximation
Han-Dong Lim, Donghwan Lee
This paper analyzes multi-step temporal difference (TD)-learning algorithms within the ``deadly triad'' scenario, characterized by linear function approximation, off-policy learnin…