Right Time to Learn:Promoting Generalization via Bio-inspired Spacing Effect in Knowledge Distillation

Guanglong Sun, Hongwei Yan, Liyuan Wang +3

Knowledge distillation (KD) is a powerful strategy for training deep neural networks (DNNs). Although it was originally proposed to train a more compact "student" model from a larg…