5 papers · 1 filter
Robust Data-Collection Policy Learning for Low-Variance Online Policy Evaluation
Claire Chen, Shuze Daniel Liu, Licheng Luo +3
In reinforcement learning policy evaluation, classic on-policy methods often suffer from high variance when estimating policy performance. To mitigate this issue, behavior policy s…
Offline Two-Player Zero-Sum Markov Games with KL Regularization
Claire Chen, Yuheng Zhang, Xinyu Liu +3
We study the problem of learning Nash equilibria in offline two-player zero-sum Markov games. While existing approaches often rely on explicit pessimism to address distribution shi…
Beyond Pessimism: Offline Learning in KL-regularized Games
Yuheng Zhang, Claire Chen, Nan Jiang
We study offline learning in KL-regularized two-player zero-sum games, where policies are optimized with respect to a fixed reference policy through KL regularization. Prior work r…
Beyond Semantic Manipulation: Token-Space Attacks on Reward Models
Yuheng Zhang, Mingyue Huo, Minghao Zhu +2
Reward models (RMs) are widely used as optimization targets in reinforcement learning from human feedback (RLHF), yet they remain vulnerable to reward hacking. Existing attacks mai…
Beyond State-Wise Mirror Descent: Offline Policy Optimization with Parametric Policies
Xiang Li, Yuheng Zhang, Nan Jiang
We investigate the theoretical aspects of offline reinforcement learning (RL) under general function approximation. While prior works (e.g., Xie et al., 2021) have established the…