5 papers
Lifted Bellman Linear Programming for Offline Reinforcement Learning
Hyukjun Yang, Jongchan Park, Narim Jeong +1
Offline reinforcement learning (RL) typically trains a critic by minimizing a regression loss against bootstrapped value targets stabilized by target networks with exponential movi…
Contraction-Aligned Analysis of Soft Bellman Residual Minimization with Weighted Lp-Norm for Markov Decision Problem
Hyukjun Yang, Han-Dong Lim, Donghwan Lee
The problem of solving Markov decision processes under function approximation remains a fundamental challenge, even under linear function approximation settings. A key difficulty a…
Periodic Regularized Q-Learning
Hyukjun Yang, Han-Dong Lim, Donghwan Lee
In reinforcement learning (RL), Q-learning is a fundamental algorithm whose convergence is guaranteed in the tabular setting. However, this convergence guarantee does not hold unde…
Bellman Residual Minimization for Control: Geometry, Stationarity, and Convergence
Donghwan Lee, Hyukjun Yang
Markov decision problems are most commonly solved via dynamic programming. Another approach is Bellman residual minimization, which directly minimizes the squared Bellman residual…
Analysis of approximate linear programming solution to Markov decision problem with log barrier function
Donghwan Lee, Hyukjun Yang, Bum Geun Park
There are two primary approaches to solving Markov decision problems (MDPs): dynamic programming based on the Bellman equation and linear programming (LP). Dynamic programming meth…