Readout Blindness: VLM Scores Miss the Spatial Direction Their Frozen Encoders Retain

Guangyuan Li, Tianming Du, Yan Jiang +2

CLIP-like vision-language models remain a cornerstone of multimodal systems, yet their scores stay near chance on directed spatial relations, such as whether one object is left of…