4 papers
Distilled Reinforcement Learning for LLM Post-training
Chen Wang, Zhaochun Li, Jionghao Bai +4
Large language model (LLM) post-training is essential for improving reasoning, adaptation, and alignment. Existing methods mainly follow two paradigms: reinforcement learning (RL)…
Implicit Compression Regularization: Concise Reasoning via Internal Shorter Distributions in RL Post-Training
Chen Wang, Hexuan Deng, Yining Zhang +5
Reinforcement learning with verifiable rewards improves LLM reasoning but often induces overthinking, where models generate unnecessarily long reasoning traces. Existing methods ma…
Towards a Theoretical Understanding to the Generalization of RLHF
Zhaochun Li, Mingyang Yi, Yue Wang +2
Reinforcement Learning from Human Feedback (RLHF) and its variants have emerged as the dominant approaches for aligning Large Language Models with human intent. While empirically e…
Distribution-Centric Policy Optimization Dominates Exploration-Exploitation Trade-off
Zhaochun Li, Chen Wang, Jionghao Bai +4
The exploration-exploitation (EE) trade-off is a central challenge in reinforcement learning (RL) for large language models (LLMs). With Group Relative Policy Optimization (GRPO),…