Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoning

Hongbin Zhang, Chaozheng Wang, Kehai Chen +4

On-policy self-distillation (OPSD) is an emerging LLM post-training paradigm in which the model serves as its own teacher: conditioned on privileged information such as a reference…