Quantifying the Knowledge in a DNN to Explain Knowledge Distillation for Classification

Quanshi Zhang, Xu Cheng, Yilan Chen +1

Compared to traditional learning from scratch, knowledge distillation sometimes makes the DNN achieve superior performance. This paper provides a new perspective to explain the suc…