When Should the Teacher Move? Temporal Coupling and Stability in Self On-Policy Distillation

Haowei Guo, Baolong Bi, Ruicheng Zhang +2

Self on-policy distillation trains a student policy against a teacher derived from its own parameter history, yet the teacher's update schedule -- which governs the \emph{temporal…