18 citations · 48 across the 19 of their papers we have counts for
5 papers · 1 filter
Policy Optimization in RLHF: The Impact of Out-of-preference Data
Ziniu Li, Tian Xu, Yang Yu
Aligning intelligent agents with human preferences and values is important. This paper examines two popular alignment methods: Direct Preference Optimization (DPO) and Reward-Model…
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models
Ziniu Li, Tian Xu, Yushun Zhang +4
Reinforcement Learning from Human Feedback (RLHF) is key to aligning Large Language Models (LLMs), typically paired with the Proximal Policy Optimization (PPO) algorithm. While PPO…
Provably Efficient Adversarial Imitation Learning with Unknown Transitions
Tian Xu, Ziniu Li, Yang Yu +1
Imitation learning (IL) has proven to be an effective method for learning good policies from expert demonstrations. Adversarial imitation learning (AIL), a subset of IL methods, is…
Deploying Offline Reinforcement Learning with Human Feedback
Ziniu Li, Ke Xu, Liu Liu +3
Reinforcement learning (RL) has shown promise for decision-making tasks in real-world applications. One practical framework involves training parameterized policy models from an of…
Theoretical Analysis of Offline Imitation With Supplementary Dataset
Ziniu Li, Tian Xu, Yang Yu +1
Behavioral cloning (BC) can recover a good policy from abundant expert data, but may fail when expert data is insufficient. This paper considers a situation where, besides the smal…