6 papers
Analysis of Off-Policy -Step TD-Learning with Linear Function Approximation
Han-Dong Lim, Donghwan Lee
This paper analyzes multi-step temporal difference (TD)-learning algorithms within the ``deadly triad'' scenario, characterized by linear function approximation, off-policy learnin…
(Im)possibility of Incentive Design for Challenge-based Blockchain Protocols
Suhyeon Lee, Dieu-Huyen Nguyen, Donghwan Lee
Blockchains offer a decentralized and secure execution environment strong enough to host cryptocurrencies, but the state-replication model makes on-chain computation expensive. To…
Periodic Regularized Q-Learning
Hyukjun Yang, Han-Dong Lim, Donghwan Lee
In reinforcement learning (RL), Q-learning is a fundamental algorithm whose convergence is guaranteed in the tabular setting. However, this convergence guarantee does not hold unde…
A finite time analysis of distributed Q-learning
Han-Dong Lim, Donghwan Lee
Multi-agent reinforcement learning (MARL) has witnessed a remarkable surge in interest, fueled by the empirical success achieved in applications of single-agent reinforcement learn…
A primal-dual perspective for distributed TD-learning
Han-Dong Lim, Donghwan Lee
The goal of this paper is to investigate distributed temporal difference (TD) learning for a networked multi-agent Markov decision process. The proposed approach is based on distri…
Understanding the theoretical properties of projected Bellman equation, linear Q-learning, and approximate value iteration
Han-Dong Lim, Donghwan Lee
In this paper, we study the theoretical properties of the projected Bellman equation (PBE) and two algorithms to solve this equation: linear Q-learning and approximate value iterat…