Adaptive Teaching with Shared Classifier for Knowledge Distillation

Jaeyeon Jang, Young-Ik Kim, Jisu Lim +1

Knowledge distillation (KD) is a technique used to transfer knowledge from an overparameterized teacher network to a less-parameterized student network, thereby minimizing the incu…