One Geometry, Different Outcomes: Readout-Dependent Effects of the Modality Gap in Vision-Language Models

Aditya Sharma, Divya Saxena

Contrastive vision-language models learn shared embedding spaces by aligning matched image-text pairs, yet their representations remain separated by a modality gap. Prior work repo…