Online Knowledge Distillation with Reward Guidance

Chen Jia

This work studies knowledge distillation (KD) for large language models (LLMs) through preference optimization. We propose a reward-guided imitation learning framework for sequenti…