Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation

Sarthak Harne, Chinmay Karkar, Yash Pandya +2

Self-distillation (SD) has emerged as a compute-efficient alternative to reinforcement learning with verifiable rewards: a self-teacher, conditioned on privileged information (PI)…