6 papers · 1 filter
RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts
Yuxin Xiong, Xunyi Jiang, Rohan Surana +8
Group Relative Policy Optimization (GRPO) has shown strong effectiveness in reinforcement learning from verifiable feedback, where sampled rollouts can be compared within a group u…
Federated Large Language Models: Current Progress and Future Directions
Yuhang Yao, Jianyi Zhang, Junda Wu +11
Large Language Models have achieved impressive performance across diverse applications, yet their training typically depends on centralized data collection, raising serious privacy…
Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning
Rohan Surana, Gagan Mundada, Xunyi Jiang +19
Reinforcement learning (RL) has become a central post-training tool for improving the reasoning abilities of large language models (LLMs). In these systems, the rollout, the trajec…
In-context Ranking Preference Optimization
Junda Wu, Rohan Surana, Zhouhang Xie +6
Recent developments in Direct Preference Optimization (DPO) allow large language models (LLMs) to function as implicit ranking models by maximizing the margin between preferred and…
Active Learning for Direct Preference Optimization
Branislav Kveton, Xintong Li, Julian McAuley +4
Direct preference optimization (DPO) is a form of reinforcement learning from human feedback (RLHF) where the policy is learned directly from preferential feedback. Although many m…
Visual Prompting in Multimodal Large Language Models: A Survey
Junda Wu, Zhehao Zhang, Yu Xia +12
Multimodal large language models (MLLMs) equip pre-trained large-language models (LLMs) with visual capabilities. While textual prompting in LLMs has been widely studied, visual pr…