2 papers
cs.LG2026
A Unified Revisit of Temperature in Classification-Based Knowledge Distillation
Logan Frank, Jim Davis
A central idea of knowledge distillation is to expose relational structure embedded in the teacher's weights for the student to learn, which is often facilitated using a temperatur…
cs.CV2025
What Makes a Good Dataset for Knowledge Distillation?
Logan Frank, Jim Davis
Knowledge distillation (KD) has been a popular and effective method for model compression. One important assumption of KD is that the teacher's original dataset will also be availa…