1 citations · 1 across the 10 of their papers we have counts for
8 papers · 1 filter
SP3O: Reinforcement Learning from Segment Preferences without Reward Modeling
Evan Assmus, Qining Zhang, Lei Ying
Preference-based reinforcement learning (PbRL) for general stochastic MDPs often requires training a reward model. Existing reward-model-free methods are either restricted to bandi…
Efficient Federated RLHF via Zeroth-Order Policy Optimization
Deyi Wang, Qining Zhang, Lei Ying
This paper considers reinforcement learning from human feedback in a federated learning setting with resource-constrained agents, such as edge devices. We propose an efficient fede…
Sign-SZPO: Provable Preference-based Reinforcement Learning with an Unknown Link Function
Qining Zhang, Lei Ying
The link function, which characterizes the relationship between the preference for two trajectories and their returns, is a crucial component in designing RL algorithms that learn…
Multi-Metric Adaptive Experimental Design Under a Fixed Budget with Validation
Qining Zhang, Tanner Fiez, Yi Liu +1
A/B tests in online experiments face statistical power challenges when testing multiple candidates simultaneously, while adaptive experimental designs (AED) alone fall short in inf…
Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inference
Qining Zhang, Lei Ying
Reward inference (learning a reward model from human preferences) is a critical intermediate step in the Reinforcement Learning from Human Feedback (RLHF) pipeline for fine-tuning…
Reinforcement Learning from Human Feedback without Reward Inference: Model-Free Algorithm and Instance-Dependent Analysis
Qining Zhang, Honghao Wei, Lei Ying
In this paper, we study reinforcement learning from human feedback (RLHF) under an episodic Markov decision process with a general trajectory-wise reward model. We developed a mode…
Cost Aware Best Arm Identification
Kellen Kanarios, Qining Zhang, Lei Ying
In this paper, we study a best arm identification problem with dual objects. In addition to the classic reward, each arm is associated with a cost distribution and the goal is to i…
Fast and Regret Optimal Best Arm Identification: Fundamental Limits and Low-Complexity Algorithms
Qining Zhang, Lei Ying
This paper considers a stochastic Multi-Armed Bandit (MAB) problem with dual objectives: (i) quick identification and commitment to the optimal arm, and (ii) reward maximization th…