Direct Preference Knowledge Distillation for Large Language Models

Yixing Li, Yuxian Gu, Li Dong +3

In the field of large language models (LLMs), Knowledge Distillation (KD) is a critical technique for transferring capabilities from teacher models to student models. However, exis…