Generalizing Teacher Networks for Effective Knowledge Distillation Across Student Architectures

Kuluhan Binici, Weiming Wu, Tulika Mitra

Knowledge distillation (KD) is a model compression method that entails training a compact student model to emulate the performance of a more complex teacher model. However, the arc…