Instance Temperature Knowledge Distillation

Zhengbo Zhang, Yuxi Zhou, Jia Gong +2

Knowledge distillation (KD) enhances the performance of a student network by allowing it to learn the knowledge transferred from a teacher network incrementally. Existing methods d…