17 citations · 44 across the 20 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2024
An Empirical Investigation into the Effect of Parameter Choices in Knowledge Distillation
Md Arafat Sultan, Aashka Trivedi, Parul Awasthy +1
We present a large-scale empirical study of how choices of configuration parameters affect performance in knowledge distillation (KD). An example of such a KD parameter is the meas…
cs.LG2023
Knowledge Distillation Label Smoothing: Fact or Fallacy?
Md Arafat Sultan
Originally proposed as a method for knowledge transfer from one model to another, some recent studies have suggested that knowledge distillation (KD) is in fact a form of regulariz…