5 papers · 1 filter
MInTRL: Off-policy Intervention can boost On-policy RL
Mingyu Chen, Yefan Tao, Gerald Friedland +2
Reinforcement learning with verifiable rewards is typically performed on-policy, keeping training data close to the current policy but limiting learning to trajectories that the po…
Accelerating RL for LLM Reasoning with Optimal Advantage Regression
Kianté Brantley, Mingyu Chen, Zhaolin Gao +4
Reinforcement learning (RL) has emerged as a powerful tool for fine-tuning large language models (LLMs) to improve complex reasoning abilities. However, state-of-the-art policy opt…
Avoiding scaling in RLHF through Preference-based Exploration
Mingyu Chen, Yiding Chen, Wen Sun +1
Reinforcement Learning from Human Feedback (RLHF) has emerged as a pivotal technique for large language model (LLM) alignment. This paper studies the setting of online RLHF and foc…
State-free Reinforcement Learning
Mingyu Chen, Aldo Pacchiano, Xuezhou Zhang
In this work, we study the \textit{state-free RL} problem, where the algorithm does not have the states information before interacting with the environment. Specifically, denote th…
Scale-free Adversarial Reinforcement Learning
Mingyu Chen, Xuezhou Zhang
This paper initiates the study of scale-free learning in Markov Decision Processes (MDPs), where the scale of rewards/losses is unknown to the learner. We design a generic algorith…