7 papers
Efficient Reinforcement Learning for Long-Horizon Tool-Use Agentic Tasks
Zelei Cheng, Amritansh Mishra, Sambit Sahu +1
Long-horizon tool-using agents must reason over user goals, domain policies, tool calls, simulator state, and delayed verifiable rewards. Reinforcement learning (RL) is a natural f…
SEAD: Competence-Aware On-Policy Distillation via Entropy-Guided Supervision
Chia-Hsuan Lee, Zelei Cheng, Yu Wang +4
On-policy distillation (OPD) has a property absent in offline distillation and RL: teacher supervision quality depends on student competence. Incoherent rollouts yield noisy gradie…
GPO: Learning from Critical Steps to Improve LLM Reasoning
Jiahao Yu, Zelei Cheng, Xian Wu +1
Large language models (LLMs) are increasingly used in various domains, showing impressive potential on different tasks. Recently, reasoning LLMs have been proposed to improve the \…
Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?
Chia-Hsuan Lee, Mingyang Zhou, Renkun Ni +6
Preference optimization methods such as DPO and KTO are widely used for aligning language models, yet little is understood about what properties of preference data drive downstream…
Building Coding Agents via Entropy-Enhanced Multi-Turn Preference Optimization
Jiahao Yu, Zelei Cheng, Xian Wu +1
Software engineering presents complex, multi-step challenges for Large Language Models (LLMs), requiring reasoning over large codebases and coordinated tool use. The difficulty of…
UC-MOA: Utility-Conditioned Multi-Objective Alignment for Distributional Pareto-Optimality
Zelei Cheng, Xin-Qiang Cai, Yuting Tang +4
Reinforcement Learning from Human Feedback (RLHF) has become a cornerstone for aligning large language models (LLMs) with human values. However, existing approaches struggle to cap…