BicKD: Bilateral Contrastive Knowledge Distillation

Jiangnan Zhu, Yukai Xu, Li Xiong +4

Knowledge distillation (KD) is a machine learning framework that transfers knowledge from a teacher model to a student model. The vanilla KD proposed by Hinton et al. has been the…