4 papers · 1 filter
From Portraits to Pathology: Probing Diagnostic Overreach in Vision-Language Models
Karan Dua, Amit Agarwal, Hitesh Laxmichand Patel +8
Many medical conditions require clinical assessment of symptoms, history, and functioning rather than judgments from appearance. We study whether vision-language models (VLMs) abst…
Do Image-Text Metrics Respect Semantic Invariances?
Amit Agarwal, Hitesh Laxmichand Patel, Meizhu Liu +9
Reference-free image-to-text evaluators are now standard for scoring image-caption alignment, yet it is unclear whether they respect semantic invariances. We present an invariance…
PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications
Hitesh Laxmichand Patel, Amit Agarwal, Srikant Panda +6
The reliability of Multimodal Large Language Models (MLLMs) in real-world settings is often undermined by sensitivity to irrelevant or distracting visual context, an aspect not cap…
RCI: A Score for Evaluating Global and Local Reasoning in Multimodal Benchmarks
Amit Agarwal, Hitesh Laxmichand Patel, Srikant Panda +7
Multimodal Large Language Models (MLLMs) have achieved impressive results on vision-language benchmarks, yet it remains unclear whether these benchmarks assess genuine global reaso…