19 citations · 27 across the 9 of their papers we have counts for
10 papers
The Verification Horizon: No Silver Bullet for Coding Agent Rewards
Binghai Wang, Chenlong Zhang, Dayiheng Liu +10
A classical intuition holds that verifying a solution is easier than producing one. For today's coding agents, this intuition is being inverted: as foundation models develop strong…
EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
Chengjun Pan, Shichun Liu, Jiahang Lin +10
Reinforcement learning (RL) for LLM post-training faces a fundamental design choice: whether to use a learned critic as a baseline for policy optimization. Classical theory favors…
MM-Doc-R1: Training Agents for Long Document Visual Question Answering through Multi-turn Reinforcement Learning
Jiahang Lin, Kai Hu, Binghai Wang +12
Conventional Retrieval-Augmented Generation (RAG) systems often struggle with complex multi-hop queries over long documents due to their single-pass retrieval. We introduce MM-Doc-…
HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning
Shenzhi Wang, Shixuan Liu, Jing Zhou +8
Vision-language models (VLMs) show strong multimodal capabilities but still struggle with fine-grained vision-language reasoning. We find that long chain-of-thought (CoT) reasoning…
Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models
Binghai Wang, Yantao Liu, Yuxuan Liu +13
Generative Reward Models (GenRMs) and LLM-as-a-Judge exhibit deceptive alignment by producing correct judgments for incorrect reasons, as they are trained and evaluated to prioriti…
AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress
Zhiheng Xi, Chenyang Liao, Guanyu Li +12
Despite rapid development, large language models (LLMs) still encounter challenges in multi-turn decision-making tasks (i.e., agent tasks) like web shopping and browser navigation,…
WorldPM: Scaling Human Preference Modeling
Binghai Wang, Runji Lin, Keming Lu +17
Motivated by scaling laws in language modeling that demonstrate how test loss scales as a power law with model and dataset sizes, we find that similar laws exist in preference mode…
RMB: Comprehensively Benchmarking Reward Models in LLM Alignment
Enyu Zhou, Guodong Zheng, Binghai Wang +11
Reward models (RMs) guide the alignment of large language models (LLMs), steering them toward behaviors preferred by humans. Evaluating RMs is the key to better aligning LLMs. Howe…
Secrets of RLHF in Large Language Models Part II: Reward Modeling
Binghai Wang, Rui Zheng, Lu Chen +24
Reinforcement Learning from Human Feedback (RLHF) has become a crucial technology for aligning language models with human values and intentions, enabling models to produce more hel…
Secrets of RLHF in Large Language Models Part I: PPO
Rui Zheng, Shihan Dou, Songyang Gao +24
Large language models (LLMs) have formulated a blueprint for the advancement of artificial general intelligence. Its primary objective is to function as a human-centric (helpful, h…