Efficient and Robust Knowledge Distillation from A Stronger Teacher Based on Correlation Matching

Wenqi Niu, Yingchao Wang, Guohui Cai +1

Knowledge Distillation (KD) has emerged as a pivotal technique for neural network compression and performance enhancement. Most KD methods aim to transfer dark knowledge from a cum…