The Hidden Uniform Cluster Prior in Self-Supervised Learning

Mahmoud Assran, Randall Balestriero, Quentin Duval +6

A successful paradigm in representation learning is to perform self-supervised pretraining using tasks based on mini-batch statistics (e.g., SimCLR, VICReg, SwAV, MSN). We show tha…