14 papers
Q-Zoom: Query-Aware Adaptive Perception for Efficient Multimodal Large Language Models
Yuheng Shi, Xiaohuan Pei, Linfeng Wen +2
MLLMs require high-resolution visual inputs for fine-grained tasks like document understanding and dense scene perception. However, current global resolution scaling paradigms indi…
Confidence Calibration under Ambiguous Ground Truth
Linwei Tao, Haoyang Luo, Minjing Dong +1
Confidence calibration assumes a unique ground-truth label per input, yet this assumption fails wherever annotators genuinely disagree. Post-hoc calibrators fitted on majority-vote…
Mitigating Object Hallucinations in Large Vision-Language Models via Attention Calibration
Younan Zhu, Linwei Tao, Minjing Dong +1
Large Vision-Language Models (LVLMs) exhibit impressive multimodal reasoning capabilities but remain highly susceptible to object hallucination, where models generate responses tha…
PA-Attack: Guiding Gray-Box Attacks on LVLM Vision Encoders with Prototypes and Attention
Hefei Mei, Zirui Wang, Chang Xu +2
Large Vision-Language Models (LVLMs) are foundational to modern multimodal applications, yet their susceptibility to adversarial attacks remains a critical concern. Prior white-box…
Catching the Details: Self-Distilled RoI Predictors for Fine-Grained MLLM Perception
Yuheng Shi, Xiaohuan Pei, Minjing Dong +1
Multimodal Large Language Models (MLLMs) require high-resolution visual information to perform fine-grained perception, yet processing entire high-resolution images is computationa…
VEAttack: Downstream-agnostic Vision Encoder Attack against Large Vision Language Models
Hefei Mei, Zirui Wang, Shen You +2
Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities in multimodal understanding and generation, yet their vulnerability to adversarial attacks raises sig…