Adaptive Explicit Knowledge Transfer for Knowledge Distillation

Hyungkeun Park, Jong-Seok Lee

Logit-based knowledge distillation (KD) for classification is cost-efficient compared to feature-based KD but often subject to inferior performance. Recently, it was shown that the…