29 citations · 37 across the 45 of their papers we have counts for
11 papers · 1 filter
A Few More Tokens Ahead: Reinforcing Parallelism in Diffusion LLMs
Jiahao Kuang, Yangfan Gao, Tao Gui +4
Parallel generation is widely regarded as an inherent advantage of diffusion large language models. Yet our fixed-quota evaluation of four representative models reveals severe degr…
Strategy Accumulation and Guided Execution for Automated LLM Fine-Tuning
Haoran Zhao, Wei Du, Dingwen Yang +7
Producing task-specific large language models requires discovering effective training strategies through experimentation. Automated fine-tuning systems have made this experimentati…
A Token-Level Analysis of Sampled-Token Reverse-KL On-Policy Distillation
Bing Shao, Jiazheng Zhang, Long Ma +12
On-policy distillation (OPD) supervises a student on its own trajectories with token-level signals from a frozen teacher, yet how a sampled loss allocates updates across tokens rem…
HSD: Hybrid Hindsight Self-Distillation
Qiye Cai, Yichuan Ma, Peiji Li +7
Reinforcement learning with verifiable rewards (RLVR) provides reliable outcome supervision for language model reasoning, but scalar trajectory rewards offer limited guidance for t…
Entropy Polarity in Reinforcement Fine-Tuning: Direction, Asymmetry, and Control
Jiazheng Zhang, Ziche Fu, Junrui Shen +17
Policy entropy has emerged as a fundamental measure for understanding and controlling exploration in reinforcement learning with verifiable rewards (RLVR) for LLMs. However, existi…
EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
Chengjun Pan, Shichun Liu, Jiahang Lin +10
Reinforcement learning (RL) for LLM post-training faces a fundamental design choice: whether to use a learned critic as a baseline for policy optimization. Classical theory favors…
Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimization
Junzhe Wang, Zhiheng Xi, Yajie Yang +4
Search agents extend Large Language Models (LLMs) beyond static parametric knowledge by enabling access to up-to-date and long-tail information unavailable during pretraining. Whil…
Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges
Xiaohua Wang, Muzhao Tian, Yuqi Zeng +20
Reinforcement Learning from Human Feedback (RLHF) and related alignment paradigms have become central to steering large language models (LLMs) and multimodal large language models…
DFPO: Scaling Value Modeling via Distributional Flow towards Robust and Generalizable LLM Post-Training
Dingwei Zhu, Zhiheng Xi, Shihan Dou +17
Training reinforcement learning (RL) systems in real-world environments remains challenging due to noisy supervision and poor out-of-domain (OOD) generalization, especially in LLM…
DVPO: Distributional Value Modeling-based Policy Optimization for LLM Post-Training
Dingwei Zhu, Zhiheng Xi, Shihan Dou +15
Reinforcement learning (RL) has shown strong performance in LLM post-training, but real-world deployment often involves noisy or incomplete supervision. In such settings, complex a…
VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training
Dingwei Zhu, Shihan Dou, Zhiheng Xi +16
Reinforcement Learning (RL) in real-world environments often suffers from ambiguous or incomplete reward supervision, which undermines policy stability and generalization. Such noi…