What Makes a "Good" Data Augmentation in Knowledge Distillation -- A Statistical Perspective

Huan Wang, Suhas Lohit, Mike Jones +1

Knowledge distillation (KD) is a general neural network training approach that uses a teacher model to guide the student model. Existing works mainly study KD from the network outp…