PURSUhInT: In Search of Informative Hint Points Based on Layer Clustering for Knowledge Distillation

Reyhan Kevser Keser, Aydin Ayanzadeh, Omid Abdollahi Aghdam +3

One of the most efficient methods for model compression is hint distillation, where the student model is injected with information (hints) from several different layers of the teac…