26 citations · 63 across the 24 of their papers we have counts for
5 papers · 1 filter
Policy Optimization in RLHF: The Impact of Out-of-preference Data
Ziniu Li, Tian Xu, Yang Yu
Aligning intelligent agents with human preferences and values is important. This paper examines two popular alignment methods: Direct Preference Optimization (DPO) and Reward-Model…
Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning
Fan-Ming Luo, Tian Xu, Xingchen Cao +1
Learning a precise dynamics model can be crucial for offline reinforcement learning, which, unfortunately, has been found to be quite challenging. Dynamics models that are learned…
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models
Ziniu Li, Tian Xu, Yushun Zhang +4
Reinforcement Learning from Human Feedback (RLHF) is key to aligning Large Language Models (LLMs), typically paired with the Proximal Policy Optimization (PPO) algorithm. While PPO…
Provably Efficient Adversarial Imitation Learning with Unknown Transitions
Tian Xu, Ziniu Li, Yang Yu +1
Imitation learning (IL) has proven to be an effective method for learning good policies from expert demonstrations. Adversarial imitation learning (AIL), a subset of IL methods, is…
Theoretical Analysis of Offline Imitation With Supplementary Dataset
Ziniu Li, Tian Xu, Yang Yu +1
Behavioral cloning (BC) can recover a good policy from abundant expert data, but may fail when expert data is insufficient. This paper considers a situation where, besides the smal…