Visualizing the embedding space to explain the effect of knowledge distillation

Hyun Seung Lee, Christian Wallraven

Recent research has found that knowledge distillation can be effective in reducing the size of a network and in increasing generalization. A pre-trained, large teacher network, for…