Spot-adaptive Knowledge Distillation

Jie Song, Ying Chen, Jingwen Ye +1

Knowledge distillation (KD) has become a well established paradigm for compressing deep neural networks. The typical way of conducting knowledge distillation is to train the studen…