Demonstrating and Reducing Shortcuts in Vision-Language Representation Learning

Maurits Bleeker, Mariya Hendriksen, Andrew Yates +1

Vision-language models (VLMs) mainly rely on contrastive training to learn general-purpose representations of images and captions. We focus on the situation when one image is assoc…