7 papers
IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals
Md. Atabuzzaman, Christian Alexander, Chris Thomas
Large Vision-Language Models (LVLMs) have achieved strong multimodal performance, yet ensuring the factual correctness of generated content remains challenging. Existing methods th…
Reliability Challenges in Diffusion Vision-Language Models
Md. Atabuzzaman, Chris Thomas
Diffusion-based Large Vision-Language Models (dLVLMs) have recently emerged as a compelling alternative to autoregressive (AR) LVLMs, offering advantages in parallel decoding, bidi…
Zero-Shot Fine-Grained Image Classification Using Large Vision-Language Models
Md. Atabuzzaman, Andrew Zhang, Chris Thomas
Large Vision-Language Models (LVLMs) have demonstrated impressive performance on vision-language reasoning tasks. However, their potential for zero-shot fine-grained image classifi…
ENTER: Event Based Interpretable Reasoning for VideoQA
Hammad Ayyubi, Junzhang Liu, Ali Asgarov +10
In this paper, we present ENTER, an interpretable Video Question Answering (VideoQA) system based on event graphs. Event graphs convert videos into graphical representations, where…
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images
Zhecan Wang, Junzhang Liu, Chia-Wei Tang +11
Existing vision-language understanding benchmarks largely consist of images of objects in their usual contexts. As a consequence, recent multimodal large language models can perfor…
Advancing Chart Question Answering with Robust Chart Component Recognition
Hanwen Zheng, Sijia Wang, Chris Thomas +1
Chart comprehension presents significant challenges for machine learning models due to the diverse and intricate shapes of charts. Existing multimodal methods often overlook these…
Detecting Multimodal Situations with Insufficient Context and Abstaining from Baseless Predictions
Junzhang Liu, Zhecan Wang, Hammad Ayyubi +5
Despite the widespread adoption of Vision-Language Understanding (VLU) benchmarks such as VQA v2, OKVQA, A-OKVQA, GQA, VCR, SWAG, and VisualCOMET, our analysis reveals a pervasive…