Distill on a Diet: Efficient Knowledge Distillation via Learnable Data Pruning

Yifan Wu, Yiqi Wang, Xichen Ye +5

Knowledge Distillation (KD) is widely used to obtain compact models for efficient inference in resource-constrained environments. Yet the computational overhead of the distillation…