Understanding Collapse in Non-Contrastive Siamese Representation Learning

Alexander C. Li, Alexei A. Efros, Deepak Pathak

Contrastive methods have led a recent surge in the performance of self-supervised representation learning (SSL). Recent methods like BYOL or SimSiam purportedly distill these contr…