Adaptive Temperature Based on Logits Correlation in Knowledge Distillation

Kazuhiro Matsuyama, Usman Anjum, Satoko Matsuyama +2

Knowledge distillation is a technique to imitate a performance that a deep learning model has, but reduce the size on another model. It applies the outputs of a model to train anot…