Simple Unsupervised Knowledge Distillation With Space Similarity

Aditya Singh, Haohan Wang

As per recent studies, Self-supervised learning (SSL) does not readily extend to smaller architectures. One direction to mitigate this shortcoming while simultaneously training a s…