An Empirical Investigation into the Effect of Parameter Choices in Knowledge Distillation

Md Arafat Sultan, Aashka Trivedi, Parul Awasthy +1

We present a large-scale empirical study of how choices of configuration parameters affect performance in knowledge distillation (KD). An example of such a KD parameter is the meas…