6 papers · 1 filter
Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration
Heyang Zhao, Xingrui Yu, David M. Bossens +2
Imitation learning is a central problem in reinforcement learning where the goal is to learn a policy that mimics the expert's behavior. In practice, it is often challenging to lea…
Variance-Dependent Regret Lower Bounds for Contextual Bandits
Jiafan He, Quanquan Gu
Variance-dependent regret bounds for linear contextual bandits, which improve upon the classical regret bound to , wher…
Logarithmic Regret for Online KL-Regularized Reinforcement Learning
Heyang Zhao, Chenlu Ye, Wei Xiong +2
Recent advances in Reinforcement Learning from Human Feedback (RLHF) have shown that KL-regularization plays a pivotal role in improving the efficiency of RL fine-tuning for large…
Towards a Sharp Analysis of Offline Policy Learning for -Divergence-Regularized Contextual Bandits
Qingyue Zhao, Kaixuan Ji, Heyang Zhao +2
Many offline reinforcement learning algorithms are underpinned by -divergence regularization, but their sample complexity *defined with respect to regularized objectives* still…
Sharp Analysis for KL-Regularized Contextual Bandits and RLHF
Heyang Zhao, Chenlu Ye, Quanquan Gu +1
Reverse-Kullback-Leibler (KL) regularization has emerged to be a predominant technique used to enhance policy optimization in reinforcement learning (RL) and reinforcement learning…
Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization
Kaixuan Ji, Guanlin Liu, Ning Dai +6
Reinforcement Learning (RL) plays a crucial role in aligning large language models (LLMs) with human preferences and improving their ability to perform complex tasks. However, curr…