What Keeps Vision-Language Models Looking at the Image?

Hiroto Osaka, Shohei Taniguchi, Gouki Minegishi +3

When do vision-language models need direct access to the image while generating an answer? We study image dependence during answer generation by examining how the visual informatio…