Decoupling Dark Knowledge via Block-wise Logit Distillation for Feature-level Alignment

Chengting Yu, Fengzhao Zhang, Ruizhe Chen +4

Knowledge Distillation (KD), a learning manner with a larger teacher network guiding a smaller student network, transfers dark knowledge from the teacher to the student via logits…