Improving Knowledge Distillation with Teacher's Explanation

Sayantan Chowdhury, Ben Liang, Ali Tizghadam +1

Knowledge distillation (KD) improves the performance of a low-complexity student model with the help of a more powerful teacher. The teacher in KD is a black-box model, imparting k…