Direct Distillation between Different Domains

Jialiang Tang, Shuo Chen, Gang Niu +4

Knowledge Distillation (KD) aims to learn a compact student network using knowledge from a large pre-trained teacher network, where both networks are trained on data from the same…