Discovering and Preserving Category Correlation Knowledge via Adaptive Reciprocal Knowledge Distillation

Dawen Jiang, Zhishu Shen, Zeyu Liu +1

Knowledge distillation aims to improve the performance of lightweight student models by transferring knowledge from larger and more powerful teacher models. However, a substantial…