TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraint

Fnu Pramono, John Cai, Sourabh Kulkarni

When visual evidence is occluded or chaotic, models should abstain. In this paper, we show that Vision-Language Models (VLMs) can internally distinguish when abstention is required…