ShiftKD: Benchmarking Knowledge Distillation under Distribution Shift

Songming Zhang, Yuxiao Luo, Ziyu Lyu +1

Knowledge Distillation (KD) transfers knowledge from large models to small models and has recently achieved remarkable success. However, the reliability of existing KD methods in r…