Know Thyself, Teach Thyself: Internal Information Flow for Selective Self-Distillation

Rui Wang, Ruijie Wang, Bo Chen +2

Self-distillation turns knowledge distillation into a closed learning loop and offers a path toward recursive self-improvement. Without an external teacher, however, the model must…