A Brief Overview: On-Policy Self-Distillation In Large Language Models

Fangming Cui, Sunan Li, Jiahong Li

On-Policy Self-Distillation (OPSD) is a unified learning framework in which a single large language model acts simultaneously as both teacher and student. Unlike conventional knowl…