On Pretraining Data Diversity for Self-Supervised Learning

Hasan Abed Al Kader Hammoud, Tuhin Das, Fabio Pizzati +3

We explore the impact of training with more diverse datasets, characterized by the number of unique samples, on the performance of self-supervised learning (SSL) under a fixed comp…