An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization

Ravid Shwartz-Ziv, Randall Balestriero, Kenji Kawaguchi +2

Variance-Invariance-Covariance Regularization (VICReg) is a self-supervised learning (SSL) method that has shown promising results on a variety of tasks. However, the fundamental m…