Distilling Knowledge from Self-Supervised Teacher by Embedding Graph Alignment

Yuchen Ma, Yanbei Chen, Zeynep Akata

Recent advances have indicated the strengths of self-supervised pre-training for improving representation learning on downstream tasks. Existing works often utilize self-supervised…