5 papers
Diagnosing the Sources of Compositional Failure in Vision-Language Models: A Controlled Analysis
Mona Gandhi, Cenk Merih Olcay, Kuan-Chieh Lo +3
Vision-language models (VLMs) often struggle with compositional reasoning tasks, but the reasons for this underperformance remain unclear. A common hypothesis is that models strugg…
Through the PRISM: Principle-Aware, Interpretable, and Multi-Scale Evaluation of Visual Designs
Mona Gandhi, KJ Joseph, Srinivasan Parthasarathy +1
Effective visual communication stems from the harmony of multiple design principles, such as readability, contrast, alignment, overlap, and coherence, which collectively govern cla…
A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis
Yue Yang, Mona Gandhi, Yufei Wang +5
While deep networks have achieved broad success in analyzing natural images, when applied to medical scans, they often fail in unexcepted situations. We investigate this challenge…
CREPE: Can Vision-Language Foundation Models Reason Compositionally?
Zixian Ma, Jerry Hong, Mustafa Omer Gul +3
A fundamental characteristic common to both human vision and natural language is their compositional nature. Yet, despite the performance gains contributed by large vision and lang…
Measuring Compositional Consistency for Video Question Answering
Mona Gandhi, Mustafa Omer Gul, Eva Prakash +3
Recent video question answering benchmarks indicate that state-of-the-art models struggle to answer compositional questions. However, it remains unclear which types of compositiona…