An Analysis of Self-supervised Pre-training with Dependent Samples

Maximilian Fleissner, Debarghya Ghoshdastidar, Samory Kpotufe

Self-supervised learning relies on so-called data augmentations $\varphi (x)$ of unlabeled datapoints $x$ --- for example, masking random pixels in an image $x$ --- that should leave the…