4 papers · 1 filter
Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning
Shijin Gong, Kai Ye, Jin Zhu +3
Recent advances in large language models (LLMs) have increasingly relied on reinforcement learning (RL) to improve their reasoning capabilities. Three types of approaches have been…
Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
Hongyi Zhou, Kai Ye, Erhan Xu +4
Group relative policy optimization (GRPO), a core methodological component of DeepSeekMath and DeepSeek-R1, has emerged as a cornerstone for scaling reasoning capabilities of large…
Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation
Hongyi Zhou, Josiah P. Hanna, Jin Zhu +2
This paper studies off-policy evaluation (OPE) in reinforcement learning with a focus on behavior policy estimation for importance sampling. Prior work has shown empirically that e…
Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach
Jin Zhu, Jingyi Li, Hongyi Zhou +3
This paper focuses on the design of spatial experiments to optimize the amount of information derived from the experimental data and enhance the accuracy of the resulting causal ef…