Representations of Text and Images Align From Layer One

Evžen Wybitul, Javier Rando, Florian Tramèr +1

We show that for a variety of concepts in adapter-based vision-language models, the representations of their images and their text descriptions are meaningfully aligned from the ve…