Does CLIP's Generalization Performance Mainly Stem from High Train-Test Similarity?

Prasanna Mayilvahanan, Thaddäus Wiedemer, Evgenia Rusak +2

Foundation models like CLIP are trained on hundreds of millions of samples and effortlessly generalize to new tasks and inputs. Out of the box, CLIP shows stellar zero-shot and few…