Rethinking Misalignment in Vision-Language Model Adaptation from a Causal Perspective

Yanan Zhang, Jiangmeng Li, Lixiang Liu +1

Foundational Vision-Language models such as CLIP have exhibited impressive generalization in downstream tasks. However, CLIP suffers from a two-level misalignment issue, i.e., task…