11 papers
Semantic-Spatial Agreement Verification for Mitigating Object Hallucination in Multimodal Large Language Models
Ziheng Ren, Qian Gao, Jun Fan +3
Multimodal large language models generate natural-language responses from visual inputs, yet may mention objects absent from an image. In medication assistance, accessible percepti…
Dual-Stream Cross-Anchor Correction Grounding Long-Form Captions and the Domain Limits of Object-Level Anchors
Lingkai Bu, Qian Gao, Jun Fan +4
Object hallucination in multimodal large language models arises when language priors and corpus co-occurrence bias outweigh the visual evidence, with nothing tying an object mentio…
Diagnosing Dense Same-Class Attribute Misbinding in Large Vision-Language Models
Yuanzhi Xu, Qian Gao, Jun Fan +4
Large vision-language models can recognize the objects and attributes in a crowded scene yet assign an attribute to the wrong same-class instance. Generic visual-question-answering…
NeurIPS: Neuro-anatomical Inductive Priors for Sphere-based Brain Decoding
Sijin Yu, Zijiao Chen, Zhenyu Yang +7
Current fMRI decoders face a performance-fidelity trade-off where efficient ID encoders outperform geometrically faithful surface-based models. We argue this is partly driven by in…
Mitigating Object Hallucinations in Vision-Language Models through Region-Aware Attention Recalibration
Yuanzhi Xu, Qian Gao, Jun Fan +4
The generation of factually incorrect objects, commonly known as object hallucination, remains a persistent challenge in Large Vision-Language Models (LVLMs). Current approaches to…
Efficient Agent: Optimizing Planning Capability for Multimodal Retrieval Augmented Generation
Yuechen Wang, Yuming Qiao, Dan Meng +4
Multimodal Retrieval-Augmented Generation (mRAG) has emerged as a promising solution to address the temporal limitations of Multimodal Large Language Models (MLLMs) in real-world s…
Improved Visual-Spatial Reasoning via R1-Zero-Like Training
Zhenyi Liao, Qingsong Xie, Yanhao Zhang +4
Increasing attention has been placed on improving the reasoning capacities of multi-modal large language models (MLLMs). As the cornerstone for AI agents that function in the physi…
Layton: Latent Consistency Tokenizer for 1024-pixel Image Reconstruction and Generation by 256 Tokens
Qingsong Xie, Zhao Zhang, Zhe Huang +3
Image tokenization has significantly advanced visual generation and multimodal modeling, particularly when paired with autoregressive models. However, current methods face challeng…
H2VU-Benchmark: A Comprehensive Benchmark for Hierarchical Holistic Video Understanding
Qi Wu, Quanlong Zheng, Yanhao Zhang +8
With the rapid development of multimodal models, the demand for assessing video understanding capabilities has been steadily increasing. However, existing benchmarks for evaluating…
X2I: Seamless Integration of Multimodal Understanding into Diffusion Transformer via Attention Distillation
Jian Ma, Qirong Peng, Xu Guo +3
Text-to-image (T2I) models are well known for their ability to produce highly realistic images, while multimodal large language models (MLLMs) are renowned for their proficiency in…
GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models
Jian Ma, Yonglin Deng, Chen Chen +3
Posters play a crucial role in marketing and advertising by enhancing visual communication and brand visibility, making significant contributions to industrial design. With the lat…