Directional Neural Collapse Explains Few-Shot Transfer in Self-Supervised Learning

Achleshwar Luthra, Yash Salunkhe, Tomer Galanti

Frozen self-supervised representations often transfer well with only a few labels across many semantic tasks. We argue that a single geometric quantity, \emph{directional} CDNV (de…