6 papers · 1 filter
Query-Adaptive Neuron Steering for LLMs
Yanxi Pan, Jingwen Wu, Yan Ma +8
Effectively eliciting the capabilities of large language models (LLMs) at inference time remains an important challenge. Existing studies address this challenge through context eng…
Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning
Shijin Gong, Kai Ye, Jin Zhu +3
Recent advances in large language models (LLMs) have increasingly relied on reinforcement learning (RL) to improve their reasoning capabilities. Three types of approaches have been…
Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
Hongyi Zhou, Kai Ye, Erhan Xu +4
Group relative policy optimization (GRPO), a core methodological component of DeepSeekMath and DeepSeek-R1, has emerged as a cornerstone for scaling reasoning capabilities of large…
Doubly Robust Alignment for Large Language Models
Erhan Xu, Kai Ye, Hongyi Zhou +3
This paper studies reinforcement learning from human feedback (RLHF) for aligning large language models with human preferences. While RLHF has demonstrated promising results, many…
Demystifying the Paradox of Importance Sampling with an Estimated History-Dependent Behavior Policy in Off-Policy Evaluation
Hongyi Zhou, Josiah P. Hanna, Jin Zhu +2
This paper studies off-policy evaluation (OPE) in reinforcement learning with a focus on behavior policy estimation for importance sampling. Prior work has shown empirically that e…
Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach
Jin Zhu, Jingyi Li, Hongyi Zhou +3
This paper focuses on the design of spatial experiments to optimize the amount of information derived from the experimental data and enhance the accuracy of the resulting causal ef…