Less is More: Selective Reflection for Compatible and Efficient Knowledge Distillation in Large Language Models

Lingyuan Liu, Mengxiang Zhang

Knowledge Distillation (KD) is a fundamental technique for compressing large language models (LLMs) into compact, efficient student models. However, existing white-box KD methods m…