Hidden in plain sight: VLMs overlook their visual representations

Stephanie Fu, Tyler Bonnen, Devin Guillory +1

Language provides a natural interface to specify and evaluate performance on visual tasks. To realize this possibility, vision language models (VLMs) must successfully integrate vi…