Student Helping Teacher: Teacher Evolution via Self-Knowledge Distillation

Zheng Li, Xiang Li, Lingfeng Yang +2

Knowledge distillation usually transfers the knowledge from a pre-trained cumbersome teacher network to a compact student network, which follows the classical teacher-teaching-stud…