4 citations · 5 across the 17 of their papers we have counts for
17 papers
RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts
Yuxin Xiong, Xunyi Jiang, Rohan Surana +8
Group Relative Policy Optimization (GRPO) has shown strong effectiveness in reinforcement learning from verifiable feedback, where sampled rollouts can be compared within a group u…
Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning
Rohan Surana, Gagan Mundada, Xunyi Jiang +19
Reinforcement learning (RL) has become a central post-training tool for improving the reasoning abilities of large language models (LLMs). In these systems, the rollout, the trajec…
Stepwise Penalization for Length-Efficient Chain-of-Thought Reasoning
Xintong Li, Sha Li, Rongmei Lin +10
Large reasoning models improve with more test-time computation, but often overthink, producing unnecessarily long chains-of-thought that raise cost without improving accuracy. Prio…
WS-GRPO: Weakly-Supervised Group-Relative Policy Optimization for Rollout-Efficient Reasoning
Gagan Mundada, Zihan Huang, Rohan Surana +8
Group Relative Policy Optimization (GRPO) is effective for training language models on complex reasoning. However, since the objective is defined relative to a group of sampled tra…
AMPS: Adaptive Modality Preference Steering via Functional Entropy
Zihan Huang, Xintong Li, Rohan Surana +5
Multimodal Large Language Models (MLLMs) often exhibit significant modality preference, which is a tendency to favor one modality over another. Depending on the input, they may ove…
SceneAlign: Aligning Multimodal Reasoning to Scene Graphs in Complex Visual Scenes
Chuhan Wang, Xintong Li, Jennifer Yuntong Zhang +5
Multimodal large language models often struggle with faithful reasoning in complex visual scenes, where intricate entities and relations require precise visual grounding at each st…
Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization
Xintong Li, Chuhan Wang, Junda Wu +4
Direct Preference Optimization (DPO) has recently been extended from text-only models to vision-language models. However, existing methods rely on oversimplified pairwise compariso…
Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamics
Sheldon Yu, Yuxin Xiong, Junda Wu +6
Recent advances in chain-of-thought (CoT) prompting have enabled large language models (LLMs) to perform multi-step reasoning. However, the explainability of such reasoning remains…
CTRLS: Chain-of-Thought Reasoning via Latent State-Transition
Junda Wu, Yuxin Xiong, Xintong Li +7
Chain-of-thought (CoT) reasoning enables large language models (LLMs) to break down complex problems into interpretable intermediate steps, significantly enhancing model transparen…
A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models
Zhouhang Xie, Junda Wu, Yiran Shen +9
Personalized preference alignment for large language models (LLMs), the process of tailoring LLMs to individual users' preferences, is an emerging research direction spanning the a…
Toward Multi-Session Personalized Conversation: A Large-Scale Dataset and Hierarchical Tree Framework for Implicit Reasoning
Xintong Li, Jalend Bantupalli, Ria Dharmani +2
There has been a surge in the use of large language models (LLM) conversational agents to generate responses based on long-term history from multiple sessions. However, existing lo…
Active Learning for Direct Preference Optimization
Branislav Kveton, Xintong Li, Julian McAuley +4
Direct preference optimization (DPO) is a form of reinforcement learning from human feedback (RLHF) where the policy is learned directly from preferential feedback. Although many m…
Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descent
Junda Wu, Yuxin Xiong, Xintong Li +9
Recent MLLMs have shown emerging visual understanding and reasoning abilities after being pre-trained on large-scale multimodal datasets. Unlike pre-training, where MLLMs receive r…
OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Models
Junda Wu, Xintong Li, Ruoyu Wang +9
Offline evaluation of LLMs is crucial in understanding their capacities, though current methods remain underexplored in existing research. In this work, we focus on the offline eva…
Visual Prompting in Multimodal Large Language Models: A Survey
Junda Wu, Zhehao Zhang, Yu Xia +12
Multimodal large language models (MLLMs) equip pre-trained large-language models (LLMs) with visual capabilities. While textual prompting in LLMs has been widely studied, visual pr…
Open-world Multi-label Text Classification with Extremely Weak Supervision
Xintong Li, Jinya Jiang, Ria Dharmani +3
We study open-world multi-label text classification under extremely weak supervision (XWS), where the user only provides a brief description for classification objectives without a…
CoMMIT: Coordinated Multimodal Instruction Tuning
Xintong Li, Junda Wu, Tong Yu +6
Instruction tuning in multimodal large language models (MLLMs) generally involves cooperative learning between a backbone LLM and a feature encoder of non-text input modalities. Th…