16 papers
RM-OPD: On-Policy Distillation for Generative Reward Models
Qinglin Su, Licheng Pan, Zhen Fang +5
Reinforcement Learning on Mixed-domain preference data (Mix-RL) has become the mainstream approach for training general-purpose Generative Reward Models (GRMs). However, Mix-RL suf…
RankBuffer: Efficient Ranking-Based Rewards for Open-Ended Generation
Zixuan Yang, Yiqun Chen, Qi Liu +6
Open-ended generation lacks canonical answers, making pointwise rewards difficult to calibrate for group-based reinforcement learning. Directly ranking same-query rollouts provides…
Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents
Qi Liu, Yiqun Chen, Zidan Chen +6
Search agents now answer questions that take dozens of searches to settle, yet how such an agent reads a page has drawn far less attention than how it finds one. Nearly all of them…
UnityMAS-O: A General RL Optimization Framework for LLM-Based Multi-Agent Systems
Yiqun Chen, Wei Yang, Erhan Zhang +14
LLM-based multi-agent systems decompose complex tasks into interacting roles, but most remain manually orchestrated by prompts, tools, and control rules, while agents are rarely op…
OASES: Outcome-Aligned Search-Evaluation Co-Training for Agentic Search
Erhan Zhang, Yiqun Chen, Zechun Niu +6
Agentic search enables language models to solve knowledge-intensive tasks by adaptively acquiring external evidence over multiple steps. Reinforcement learning with verifiable rewa…
Knowledge-Graph Paths as Intermediate Supervision for Self-Evolving Search Agents
Huyu Wu, Jun Liu, Xiaochi Wei +3
Self-evolving search agents reduce reliance on human-written training questions by generating and solving their own search tasks. We build on Search Self-Play (SSP), a representati…
Tournament-GRPO: Group-Wise Tournament Rewards for Reinforcement Learning in Open-Ended Long-Form Generation
Zixuan Yang, Yiqun Chen, Wei Yang +7
Reinforcement learning in open-ended long-form generation is challenging because reliable reference answers and automatic metrics are often unavailable. Existing rubric-based metho…
OPRIDE: Offline Preference-based Reinforcement Learning via In-Dataset Exploration
Yiqin Yang, Hao Hu, Yihuan Mao +10
Preference-based reinforcement learning (PbRL) can help avoid sophisticated reward designs and align better with human intentions, showing great promise in various real-world appli…
Self-Compression of Chain-of-Thought via Multi-Agent Reinforcement Learning
Yiqun Chen, Jinyuan Feng, Wei Yang +9
The inference overhead induced by redundant reasoning undermines the interactive experience and severely bottlenecks the deployment of Large Reasoning Models. Existing reinforcemen…
JADE: Bridging the Strategic-Operational Gap in Dynamic Agentic RAG
Yiqun Chen, Erhan Zhang, Tianyi Hu +8
The evolution of Retrieval-Augmented Generation (RAG) has shifted from static retrieval pipelines to dynamic, agentic workflows where a central planner orchestrates multi-turn reas…
Owen-Shapley Policy Optimization: A Principled RL Algorithm for Generative Search LLMs
Abhijnan Nath, Alireza Bagheri Garakani, Tianchen Zhou +3
Large language models are increasingly trained via reinforcement learning for personalized recommendation tasks, but standard methods like GRPO rely on sparse, sequence-level rewar…
Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models
Wei Wu, Liyi Chen, Congxi Xiao +7
Large reasoning models enhanced by reinforcement learning with verifiable rewards have achieved significant performance gains by extending their chain-of-thought. However, this par…
RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering
Rongyang Zhang, Yuqing Huang, Chengqiang Lu +8
In real-world scenarios, providing user queries with visually enhanced responses can considerably benefit understanding and memory, underscoring the great value of interleaved imag…
SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment
Yuqing Huang, Rongyang Zhang, Qimeng Wang +9
Recent advancements in large language models (LLMs) have revolutionized natural language processing through their remarkable capabilities in understanding and executing diverse tas…
PaRT: Enhancing Proactive Social Chatbots with Personalized Real-Time Retrieval
Zihan Niu, Zheyong Xie, Shaosheng Cao +9
Social chatbots have become essential intelligent companions in daily scenarios ranging from emotional support to personal interaction. However, conventional chatbots with passive…
Qilin: A Multimodal Information Retrieval Dataset with APP-level User Sessions
Jia Chen, Qian Dong, Haitao Li +9
User-generated content (UGC) communities, especially those featuring multimodal content, improve user experiences by integrating visual and textual information into results (or ite…