6 papers
FADE: Mitigating Hallucinations by Reducing Language-Prior Dominance in Large Vision-Language Models
Yichen Guo, Kai Tang, Fenglai Lin +5
Despite the impressive capabilities of Large Vision-Language Models (LVLMs), they remain susceptible to hallucination, generating content inconsistent with the input image. Recent…
TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference
Tinghao Wang, Yichen Guo, Rui Huang +11
Multimodal large language models (MLLMs) have achieved strong multimodal reasoning capabilities, but their efficiency is limited by the large number of visual tokens, which introdu…
Mitigating Context-Memory Conflicts in LLMs through Dynamic Cognitive Reconciliation Decoding
Yigeng Zhou, Wu Li, Yifan Lu +6
Large language models accumulate extensive parametric knowledge through pre-training. However, knowledge conflicts occur when outdated or incorrect parametric knowledge conflicts w…
Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognition
Hui Liu, Kecheng Chen, Jialiang Wang +3
Vision-Language Models (VLMs), such as CLIP, have significantly advanced zero-shot image recognition. However, their performance remains limited by suboptimal prompt engineering an…
Enhancing Zero-Shot Image Recognition in Vision-Language Models through Human-like Concept Guidance
Hui Liu, Wenya Wang, Kecheng Chen +6
In zero-shot image recognition tasks, humans demonstrate remarkable flexibility in classifying unseen categories by composing known simpler concepts. However, existing vision-langu…
Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning
Hui Liu, Wenya Wang, Hao Sun +4
Large Language Models (LLMs) have demonstrated impressive in-context learning (ICL) capabilities from few-shot demonstration exemplars. While recent learning-based demonstration se…