13 papers
Training LLM Judges from Language Feedback via Position-Selective Self-Distillation
Ilgee Hong, Changlong Yu, Zhenghao Xu +5
We study training LLM judges from natural language feedback, especially for subjective tasks where the verdict depends strongly on which evaluation criteria the judge invokes and h…
HierDoc: Hierarchical Page-to-Region Evidence Routing for Long-Document Visual Question Answering
Rongjian Gu, Wengang Zhou, Junyu Xiong +4
Multi-page document visual question answering requires locating sparse evidence at both the page and region levels. Existing approaches typically emphasize one level over the other…
Hierarchical Evidence-Driven Reasoning for Long Document Understanding
Junyu Xiong, Yonghui Wang, Rongjian Gu +4
Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset. However, existi…
Customer-Agent: Overcoming Context Limitations in Ultra-Long Shopping Trajectories via Tool-Augmented Agents and RLVR
Hongye Liu, Rongmei Lin, Anurag Kashyap +4
Understanding customer shopping trajectories is essential for enabling personalized shopping experiences. However, shopping records (i.e., customer's search, clicks, purchases, etc…
Unlocking Latent Value: Taxonomy-Guided Recovery of High-Performing Data from Low-Tier Web Corpora
Neeraj Varshney, Sanket Lokegaonkar, Nasser Zalmout +3
Dominant web data curation pipelines for pretraining collapse document quality into a single composite score, systematically missing high-value content along dimensions the scorer…
CoMem: Context Management with A Decoupled Long-Context Model
Yuwei Zhang, Chengyu Dong, Shuowei Jin +11
Context management enables agentic models to solve long-horizon tasks through iterative summarization of previous interaction histories. However, this process typically incurs subs…
Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation
Yuwei Zhang, Sha Li, Changlong Yu +9
Enabling Large Language Models (LLMs) to continuously improve from environmental interactions is a central challenge in post-training. While on-policy self-distillation offers a pr…
Self-Rewarding PPO: Aligning Large Language Models with Demonstrations Only
Qingru Zhang, Liang Qiu, Ilgee Hong +11
Supervised fine-tuning (SFT) has emerged as a crucial method for aligning large language models (LLMs) with human-annotated demonstrations. However, SFT, being an off-policy approa…
Improving Sampling Efficiency in RLVR through Adaptive Rollout and Response Reuse
Yuheng Zhang, Wenlin Yao, Changlong Yu +5
Large language models (LLMs) have achieved impressive reasoning performance, with reinforcement learning with verifiable rewards (RLVR) emerging as a standard paradigm for post-tra…
DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding
Junyu Xiong, Yonghui Wang, Weichao Zhao +4
Understanding multi-page documents poses a significant challenge for multimodal large language models (MLLMs), as it requires fine-grained visual comprehension and multi-hop reason…
AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs
Nicholas E. Corrado, Julian Katz-Samuels, Adithya Devraj +6
When aligning large language models (LLMs), their performance on various tasks (such as being helpful, harmless, and honest) depends heavily on the composition of their training da…
WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning
Zhepei Wei, Wenlin Yao, Yao Liu +9
While reinforcement learning (RL) has demonstrated remarkable success in enhancing large language models (LLMs), it has primarily focused on single-turn tasks such as solving math…
M-LLM Based Video Frame Selection for Efficient Video Understanding
Kai Hu, Feng Gao, Xiaohan Nie +8
Recent advances in Multi-Modal Large Language Models (M-LLMs) show promising results in video reasoning. Popular Multi-Modal Large Language Model (M-LLM) frameworks usually apply n…