Not Every Divergence Should Be Suppressed: Counterfactual Recoverability in On-Policy Distillation

De Jiang, Zhengyang Zhang, Kehong Yuan +1

On-policy distillation (OPD) supervises student-visited trajectories, yet divergence-based rules cannot determine whether an erroneous prefix remains correctable. We formulate this…