One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks

Gregor Geigle, Chen Cecilia Liu, Jonas Pfeiffer +1

Current multimodal models, aimed at solving Vision and Language (V+L) tasks, predominantly repurpose Vision Encoders (VE) as feature extractors. While many VEs -- of different arch…