Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning

Zixin Wen, Yuanzhi Li

How can neural networks trained by contrastive learning extract features from the unlabeled data? Why does contrastive learning usually need much stronger data augmentations than s…