6 papers
MLLMs Get It Right, Then Get It Wrong: Tracing and Correcting Late-Layer Textual Bias
Xingming Li, Ao Cheng, Qiyao Sun +4
When vision contradicts text, multimodal large language models (MLLMs) consistently favor text, even when images provide clear evidence otherwise. This bias poses risks for applica…
Advantage Collapse in Group Relative Policy Optimization: Diagnosis and Mitigation
Xixiang He, Qiyao Sun, Ao Cheng +5
Group Relative Policy Optimization (GRPO), a prominent algorithm within the Reinforcement Learning from Verifiable Rewards (RLVR) framework, has achieved strong results in improvin…
StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning
Xixiang He, Baiqi Wu, Xingming Li +4
Multimodal large language models (MLLMs) often know the rule but pick the wrong answer: on abstract visual reasoning (AVR) tasks, a model can describe what it sees and name the und…
Efficient Hallucination Detection: Adaptive Bayesian Estimation of Semantic Entropy with Guided Semantic Exploration
Qiyao Sun, Xingming Li, Xixiang He +5
Large language models (LLMs) have achieved remarkable success in various natural language processing tasks, yet they remain prone to generating factually incorrect outputs known as…
ENC-Bench: A Benchmark for Evaluating Multimodal Large Language Models in Electronic Navigational Chart Understanding
Ao Cheng, Xingming Li, Xuanyu Ji +5
Electronic Navigational Charts (ENCs) are the safety-critical backbone of modern maritime navigation, yet it remains unclear whether multimodal large language models (MLLMs) can re…
TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection
Xixiang He, Hao Yu, Qiyao Sun +4
Instruction Fine-Tuning (IFT) is crucial for aligning large language models (LLMs) with human preferences, and selecting a small yet representative subset from massive data signifi…