What Do Students Learn? A Feature-Level Analysis of Dark Knowledge

Seungu Kang, Songkuk Kim

Knowledge Distillation (KD) is a powerful tool for model compression, yet the precise mechanisms by which student models acquire feature representations remain underexplored. In th…