2 papers
cs.CV2025
Crossmodal Knowledge Distillation with WordNet-Relaxed Text Embeddings for Robust Image Classification
Chenqi Guo, Mengshuo Rong, Qianli Feng +2
Crossmodal knowledge distillation (KD) aims to enhance a unimodal student using a multimodal teacher model. In particular, when the teacher's modalities include the student's, addi…
cs.LG2024
Why does Knowledge Distillation Work? Rethink its Attention and Fidelity Mechanism
Chenqi Guo, Shiwei Zhong, Xiaofeng Liu +2
Does Knowledge Distillation (KD) really work? Conventional wisdom viewed it as a knowledge transfer procedure where a perfect mimicry of the student to its teacher is desired. Howe…