Single-Teacher View Augmentation: Enhancing Knowledge Distillation with Student-Guided Perturbations

Xuyi Yu, Yaohua Liu, Chengjun Li +6

Knowledge distillation (KD) typically relies on the fixed perspective of a single teacher, limiting the diversity of supervisory signals. While multi-teacher distillation addresses…