An Investigation into Whitening Loss for Self-supervised Learning

Xi Weng, Lei Huang, Lei Zhao +3

A desirable objective in self-supervised learning (SSL) is to avoid feature collapse. Whitening loss guarantees collapse avoidance by minimizing the distance between embeddings of…