$\mathcal{X}$-KD: General Experiential Knowledge Distillation for Large Language Models

Yuang Cai, Yuyu Yuan

Knowledge Distillation (KD) for Large Language Models (LLMs) has become increasingly important as models grow in size and complexity. While existing distillation approaches focus o…