Knowing When Not to Answer: Abstention and Refusal Reasoning in Vision--Language Models

Karan Dua, Amit Agarwal, Hitesh Laxmichand Patel +8

Many medical conditions require diagnosis through detailed, multi-context clinical assessment rather than from visual appearance alone. Despite this, vision-language models (VLMs)…