From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance

Maximilian Dreyer, Lorenz Hufe, Jim Berend +3

Transformer-based CLIP models are widely used for text-image probing and feature extraction, making it relevant to understand the internal mechanisms behind their predictions. Whil…