6 papers
Compact-Bench: Measuring What Context Compaction Costs Long-Horizon Coding Agents
Ruopei Sun, Jianfeng Cai, Kangwen Zhao +2
Context compaction lets long-horizon coding agents continue beyond their context windows, but can erase the state needed to finish a task. Prior evaluations measure efficiency, ret…
Dual-Axis Policy Optimization for LLM Agents: Bayesian Feedback Attribution and Trajectory Mass Normalization
Yingxuan Zhuang, Binhe Yu, Jingxiao Yang +6
Reinforcement learning for LLM agents involves two distinct optimization di- mensions: how environment feedback is exploited within a trajectory, and how complete trajectories are…
CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation
Jianfeng Cai, Jinhua Zhu, Ruopei Sun +5
The rise of reasoning models necessitates large-scale verifiable data, for which programming tasks serve as an ideal source. However, while competitive programming platforms provid…
Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks
Ruopei Sun, Jianfeng Cai, Jinhua Zhu +5
RLHF has emerged as a predominant approach for aligning artificial intelligence systems with human preferences, demonstrating exceptional and measurable efficacy in instruction fol…
Bias Fitting to Mitigate Length Bias of Reward Model in RLHF
Kangwen Zhao, Jianfeng Cai, Jinhua Zhu +5
Reinforcement Learning from Human Feedback (RLHF) relies on reward models to align large language models with human preferences. However, RLHF often suffers from reward hacking, wh…
Disentangling Length Bias In Preference Learning Via Response-Conditioned Modeling
Jianfeng Cai, Jinhua Zhu, Ruopei Sun +4
Reinforcement Learning from Human Feedback (RLHF) has achieved considerable success in aligning large language models (LLMs) by modeling human preferences with a learnable reward m…