11 papers
Why Does Feedback-Augmented Self-Distillation Fail to Improve Retrieval-Interleaved Search Agents?
Fan Yang, Rui Meng, Yuxin Wen
On-policy self-distillation (OPSD) offers a promising approach for training large language models without relying on a separate teacher model. However, its effectiveness on complex…
KEPO: Knowledge-Enhanced Preference Optimization for Multimodal Reasoning with Applications to Medical VQA
Fan Yang, Rui Meng, Trudi Di Qi +2
Reinforcement learning (RL) has emerged as a promising paradigm for inducing explicit reasoning behaviors in large language and vision-language models. However, reasoning-oriented…
Beyond Position Bias: Shifting Context Compression from Position-Driven to Semantic-Driven
Jiwei Tang, Zhijing Huang, Xinyu Zhang +5
Large Language Models (LLMs) have demonstrated exceptional performance across diverse tasks. However, their deployment in long-context scenarios faces high computational overhead a…
Breaking the Batch Barrier (B3) of Contrastive Learning via Smart Batch Mining
Raghuveer Thirukovalluru, Rui Meng, Ye Liu +7
Contrastive learning (CL) is a prevalent technique for training embedding models, which pulls semantically similar examples (positives) closer in the representation space while pus…
Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknown
Lifu Tu, Rui Meng, Shafiq Joty +2
Large language models (LLMs) have demonstrated strong capabilities in text understanding and generation. However, they often lack factuality, producing a mixture of true and false…
CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval
Ye Liu, Rui Meng, Shafiq Joty +4
Despite the success of text retrieval in many NLP tasks, code retrieval remains a largely underexplored area. Most text retrieval systems are tailored for natural language queries,…