Feature diversity in self-supervised learning

Pranshu Malviya, Arjun Vaithilingam Sudhakar

Many studies on scaling laws consider basic factors such as model size, model shape, dataset size, and compute power. These factors are easily tunable and represent the fundamental…