Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Information

Chanho Park, Daehyeon Choi, Jihyun Lee +1

Vision-language models (VLMs) can locate an image region referred to by a text prompt and route the corresponding visual evidence to the output, yet the internal mechanism behind t…