Boosting Contrastive Learning with Relation Knowledge Distillation

Kai Zheng, Yuanjiang Wang, Ye Yuan

While self-supervised representation learning (SSL) has proved to be effective in the large model, there is still a huge gap between the SSL and supervised method in the lightweigh…