From Knowing to Abstaining: Bridging the Representation-Action Gap in Vision-Language Models

Jialuo He, Huangxun Chen

The ability of vision-language models (VLMs) to abstain from unanswerable questions is as important as their ability to answer answerable ones accurately. Recently, several benchma…