iCD: A Implicit Clustering Distillation Mathod for Structural Information Mining

Xiang Xue, Yatu Ji, Qing-dao-er-ji Ren +6

Logit Knowledge Distillation has gained substantial research interest in recent years due to its simplicity and lack of requirement for intermediate feature alignment; however, it…