4 papers
See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs
Yuqing Lei, Wenbo Lyu, Yingjun Du +3
Large Vision-Language Models (LVLMs) excel at multimodal tasks but remain prone to object hallucinations. Prior training-free remedies often uniformly strengthen visual signals, wh…
MetaTPT: Meta Test-time Prompt Tuning for Vision-Language Models
Yuqing Lei, Yingjun Du, Yawen Huang +2
Vision-language models (VLMs) such as CLIP exhibit strong zero-shot generalization but remain sensitive to domain shifts at test time. Test-time prompt tuning (TPT) mitigates this…
VisChainBench: A Benchmark for Multi-Turn, Multi-Image Visual Reasoning Beyond Language Priors
Wenbo Lyu, Yingjun Du, Jinglin Zhao +2
Understanding multi-image, multi-turn scenarios is a critical yet underexplored capability for Large Vision-Language Models (LVLMs). Existing benchmarks predominantly focus on stat…
Variational Task Vector Composition
Boyuan Zhang, Yingjun Du, Xiantong Zhen +1
Task vectors capture how a model changes during fine-tuning by recording the difference between pre-trained and task-specific weights. The composition of task vectors, a key operat…