Swing Distillation: A Privacy-Preserving Knowledge Distillation Framework

Junzhuo Li, Xinwei Wu, Weilong Dong +3

Knowledge distillation (KD) has been widely used for model compression and knowledge transfer. Typically, a big teacher model trained on sufficient data transfers knowledge to a sm…