Ill-Posed by Design: Probing Evidence Use in VLMs

Boaz Meivar, Shaked Perek, Shani Shvartzman +2

Counterfactual analysis is widely used to study evidence use in vision-language models, but its diagnostic value is limited on well-posed tasks: when several cues independently sup…