Modulate Your Spectrum in Self-Supervised Learning

Xi Weng, Yunhao Ni, Tengwei Song +5

Whitening loss offers a theoretical guarantee against feature collapse in self-supervised learning (SSL) with joint embedding architectures. Typically, it involves a hard whitening…