Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping

Yijie Chen, Yijin Liu, Fandong Meng +3

Knowledge Distillation (KD) has emerged as a prominent technique for model compression. However, conventional KD approaches primarily focus on homogeneous architectures with identi…