CKD: Contrastive Knowledge Distillation from A Sample-wise Perspective

Wencheng Zhu, Xin Zhou, Pengfei Zhu +2

In this paper, we propose a simple yet effective contrastive knowledge distillation framework that achieves sample-wise logit alignment while preserving semantic consistency. Conve…