Multi-level Knowledge Distillation via Knowledge Alignment and Correlation

Fei Ding, Yin Yang, Hongxin Hu +2

Knowledge distillation (KD) has become an important technique for model compression and knowledge transfer. In this work, we first perform a comprehensive analysis of the knowledge…