8 papers
Regularized Emphatic Temporal-Difference Learning: Stability under Constant Stepsizes
Xingguo Chen, Zhaohui Wu, Jinguo Ye +5
Emphatic temporal-difference learning (ETD) stabilizes the expected off-policy TD update and changes its projection geometry, but neither property determines constant-stepsize samp…
Behavior-Aware Auxiliary Corrections for Off-Policy Temporal-Difference Prediction
Xingguo Chen, Zhiang He, Yuchen Shen +4
Temporal-difference learning with function approximation can be unstable under off-policy sampling. TDC stabilizes off-policy TD through an auxiliary covariance correction, and TDR…
Behavior-Induced Mirror-Prox Temporal-Difference Learning for Faster Off-Policy Prediction
Xingguo Chen, Yuchen Shen, Shangdong Yang +3
Gradient temporal-difference methods provide stable off-policy prediction with linear function approximation, but their practical performance is strongly affected by the geometry i…
Regularized Centered Emphatic Temporal Difference Learning
Xingguo Chen, Chaohui Wu, Jinguo Ye +5
Off-policy temporal-difference (TD) learning with function approximation faces a structural tradeoff among stability, projection geometry, and variance control. Emphatic TD (ETD) i…
Bitboard version of Tetris AI
Xingguo Chen, Pingshou Xiong, Zhenyu Luo +6
The efficiency of game engines and policy optimization algorithms is crucial for training reinforcement learning (RL) agents in complex sequential decision-making tasks, such as Te…
Bellman Error Centering
Xingguo Chen, Yu Gong, Shangdong Yang +1
This paper revisits the recently proposed reward centering algorithms including simple reward centering (SRC) and value-based reward centering (VRC), and points out that SRC is ind…
A Variance Minimization Approach to Temporal-Difference Learning
Xingguo Chen, Yu Gong, Shangdong Yang +1
Fast-converging algorithms are a contemporary requirement in reinforcement learning. In the context of linear function approximation, the magnitude of the smallest eigenvalue of th…
Model-based Offline Policy Optimization with Adversarial Network
Junming Yang, Xingguo Chen, Shengyuan Wang +1
Model-based offline reinforcement learning (RL), which builds a supervised transition model with logging dataset to avoid costly interactions with the online environment, has been…