7 papers
Process Advantage Signal Shaping: A Paradigm-Agnostic Middleware for Process-Supervised RL in LLM Reasoners
Chao Wang, Hongtao Tian, Tao Yang +3
Group Relative Policy Optimization (GRPO) is a default recipe for process-supervised reinforcement learning of LLM reasoners, and dense process supervision -- via learned process r…
CAPO: Towards Enhancing LLM Reasoning through Generative Credit Assignment
Guofu Xie, Yunsheng Shi, Hongtao Tian +2
Reinforcement Learning with Verifiable Rewards (RLVR) has improved the reasoning abilities of Large Language Models (LLMs) by using rule-based binary feedback. However, current RLV…
Merge and Guide: Unifying Model Merging and Guided Decoding for Controllable Multi-Objective Generation
Guofu Xie, Chen Zhang, Xiao Zhang +3
Adapting to diverse user needs at test time is a key challenge in controllable multi-objective generation. Existing methods are insufficient: merging-based approaches provide indir…
From Faithfulness to Correctness: Generative Reward Models that Think Critically
Qiyao Ma, Yunsheng Shi, Hongtao Tian +3
Through reinforcement learning with verifiable rewards (RLVR), large language models have achieved substantial progress in domains with easily verifiable outcomes, such as mathemat…
Learning More with Less: A Dynamic Dual-Level Down-Sampling Framework for Efficient Policy Optimization
Chao Wang, Tao Yang, Hongtao Tian +5
Critic-free methods like GRPO reduce memory demands by estimating advantages from multiple rollouts but tend to converge slowly, as critical learning signals are diluted by an abun…
WeChat-YATT: A Scalable, Simple, Efficient, and Production Ready Training Library
Junyu Wu, Weiming Chang, Xiaotao Liu +10
Reinforcement Learning from Human Feedback (RLHF) has emerged as a prominent paradigm for training large language models and multimodal systems. Despite the notable advances enable…