When are Lemons Purple? The Concept Association Bias of Vision-Language Models

Yutaro Yamada, Yingtian Tang, Yoyo Zhang +1

Large-scale vision-language models such as CLIP have shown impressive performance on zero-shot image classification and image-to-text retrieval. However, such performance does not…