6 papers · 1 filter
Robust Data-Collection Policy Learning for Low-Variance Online Policy Evaluation
Claire Chen, Shuze Daniel Liu, Licheng Luo +3
In reinforcement learning policy evaluation, classic on-policy methods often suffer from high variance when estimating policy performance. To mitigate this issue, behavior policy s…
Offline Two-Player Zero-Sum Markov Games with KL Regularization
Claire Chen, Yuheng Zhang, Xinyu Liu +3
We study the problem of learning Nash equilibria in offline two-player zero-sum Markov games. While existing approaches often rely on explicit pessimism to address distribution shi…
Beyond Semantic Manipulation: Token-Space Attacks on Reward Models
Yuheng Zhang, Mingyue Huo, Minghao Zhu +2
Reward models (RMs) are widely used as optimization targets in reinforcement learning from human feedback (RLHF), yet they remain vulnerable to reward hacking. Existing attacks mai…
Beyond State-Wise Mirror Descent: Offline Policy Optimization with Parametric Policies
Xiang Li, Yuheng Zhang, Nan Jiang
We investigate the theoretical aspects of offline reinforcement learning (RL) under general function approximation. While prior works (e.g., Xie et al., 2021) have established the…
Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs
Yuheng Zhang, Nan Jiang
We investigate off-policy evaluation (OPE), a central and fundamental problem in reinforcement learning (RL), in the challenging setting of Partially Observable Markov Decision Pro…
On the Curses of Future and History in Future-dependent Value Functions for Off-policy Evaluation
Yuheng Zhang, Nan Jiang
We study off-policy evaluation (OPE) in partially observable environments with complex observations, with the goal of developing estimators whose guarantee avoids exponential depen…