Self Regulated Learning Mechanism for Data Efficient Knowledge Distillation

Sourav Mishra, Suresh Sundaram

Existing methods for distillation do not efficiently utilize the training data. This work presents a novel approach to perform distillation using only a subset of the training data…