Enhancing VICReg: Random-Walk Pairing for Improved Generalization and Better Global Semantics Capturing

Idan Simai, Ronen Talmon, Uri Shaham

In this paper, we argue that viewing VICReg-a popular self-supervised learning (SSL) method--through the lens of spectral embedding reveals a potential source of sub-optimality: it…