8 papers
Training LLM Judges from Language Feedback via Position-Selective Self-Distillation
Ilgee Hong, Changlong Yu, Zhenghao Xu +5
We study training LLM judges from natural language feedback, especially for subjective tasks where the verdict depends strongly on which evaluation criteria the judge invokes and h…
Process Reward Informed Tree Rollout for Effective Multi-Turn RL
Xintong Li, Sha Li, Yuwei Zhang +8
Reinforcement learning (RL) has become a key approach for training LLM agents, yet popular methods such as GRPO/RLOO rely on multiple independently sampled complete trajectories fo…
SafeGEO: Understanding Generative Engine Optimization Risks in Recommendation Agents
Qianfeng Wen, Yifan Simon Liu, Xin Liu +4
Generative Engine Optimization (GEO) lets content owners rewrite web content to increase their visibility in generative systems. In recommendation agents, this creates a risk that…
Recovering Policy-Induced Errors: Benchmarking and Trajectory Synthesis for Robust GUI Agents
Tianpeng Bu, Xin Liu, Qihua Chen +7
While GUI agents have advanced rapidly, they often lack the robustness to recover from their own errors, hindering real-world deployment. To bridge this gap at both the evaluation…
Long Live The Balance: Information Bottleneck Driven Tree-based Policy Optimization
Hao Jiang, Shurui Li, Tianpeng Bu +7
Recent advances in online reinforcement learning (RL) for large language models (LLMs) have demonstrated promising performance in complex reasoning tasks. However, they often exhib…
Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
Yining Lu, Zilong Wang, Shiyang Li +6
Prior work in multi-objective reinforcement learning typically uses linear reward scalarization with fixed weights, which provably fails to capture non-convex Pareto fronts and thu…
Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models
Ilgee Hong, Changlong Yu, Liang Qiu +8
Reinforcement learning from human feedback (RLHF) has become a powerful post-training paradigm for aligning large language models with human preferences. A core challenge in RLHF i…
Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data
Siqi Guo, Ilgee Hong, Vicente Balmaseda +6
Supervised fine-tuning (SFT) has become a crucial step for aligning pretrained large language models (LLMs) using supervised datasets of input-output pairs. However, despite being…