6 citations · 15 across the 11 of their papers we have counts for
5 papers · 1 filter
Scale Can't Overcome Pragmatics: The Impact of Reporting Bias on Vision-Language Reasoning
Amita Kamath, Jack Hessel, Khyathi Chandu +3
The lack of reasoning capabilities in Vision-Language Models (VLMs) has remained at the forefront of research discourse. We posit that this behavior stems from a reporting bias in…
The Hard Positive Truth about Vision-Language Compositionality
Amita Kamath, Cheng-Yu Hsieh, Kai-Wei Chang +1
Several benchmarks have concluded that our best vision-language models (e.g., CLIP) are lacking in compositionality. Given an image, these benchmarks probe a model's ability to ide…
What's "up" with vision-language models? Investigating their struggle with spatial reasoning
Amita Kamath, Jack Hessel, Kai-Wei Chang
Recent vision-language (VL) models are powerful, but can they reliably distinguish "right" from "left"? We curate three new corpora to quantify model comprehension of such basic sp…
Text encoders bottleneck compositionality in contrastive vision-language models
Amita Kamath, Jack Hessel, Kai-Wei Chang
Performant vision-language (VL) models like CLIP represent captions using a single vector. How much information about language is lost in this bottleneck? We first curate CompPromp…
Selective Question Answering under Domain Shift
Amita Kamath, Robin Jia, Percy Liang
To avoid giving wrong answers, question answering (QA) models need to know when to abstain from answering. Moreover, users often ask questions that diverge from the model's trainin…