A Statistical Theory of Contrastive Learning via Approximate Sufficient Statistics

Licong Lin, Song Mei

Contrastive learning -- a modern approach to extract useful representations from unlabeled data by training models to distinguish similar samples from dissimilar ones -- has driven…