Overconfident and Blind to Details: Fixing Prompt Insensitivity with Abductive Preference Learning

Yijin Ni, Simon Yu, Peng Qi

Vision and language models frequently ignore semantically critical input edits, defaulting to pretraining priors. For example, models will confidently assert a five-legged dog has…