Reevaluating the Intra-Modal Misalignment Hypothesis in CLIP

Jonas Herzog, Yue Wang

Recent research suggested that the embeddings produced by CLIP-like contrastive language-image training are suboptimal for image-only tasks. The main theory is that the inter-modal…