5 papers
OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning
Yu Li, Rui Miao, Tian Lan +1
Reinforcement learning with verifiable rewards has become the standard recipe for improving LLM reasoning, but the dominant algorithm GRPO assigns a single trajectory-level advanta…
ARISE: Agent Reasoning with Intrinsic Skill Evolution in Hierarchical Reinforcement Learning
Yu Li, Rui Miao, Zhengling Qi +1
The dominant paradigm for improving mathematical reasoning in language models relies on Reinforcement Learning with verifiable rewards. Yet existing methods treat each problem inst…
A Principled Path to Fitted Distributional Evaluation
Sungee Hong, Jiayi Wang, Zhengling Qi +1
In reinforcement learning, distributional off-policy evaluation (OPE) focuses on estimating the return distribution of a target policy using offline data collected under a differen…
Distributional Off-policy Evaluation with Bellman Residual Minimization
Sungee Hong, Zhengling Qi, Raymond K. W. Wong
We study distributional off-policy evaluation (OPE), of which the goal is to learn the distribution of the return for a target policy using offline data generated by a different po…
A Fine-grained Analysis of Fitted Q-evaluation: Beyond Parametric Models
Jiayi Wang, Zhengling Qi, Raymond K. W. Wong
In this paper, we delve into the statistical analysis of the fitted Q-evaluation (FQE) method, which focuses on estimating the value of a target policy using offline data generated…