VISTA: Verifier-Informed Student-to-Teacher Adaptation for On-Policy Self-Distillation

Zewen Ding, Zezhong Wu, Zhou Tao +5

On-policy self-distillation (OPSD) improves reasoning by training a problem-only student on its own rollouts using dense token-level supervision from a privileged teacher that also…