Is More Privileged Information Better? From Solution Traces to Problem-Solving Structure in Self-Distilled Reasoning

Xuyang Zhao, Liting Zhang, Zichen Xu +4

On-policy self-distillation (OPSD) improves reasoning by using a privileged view of a model conditioned on reference solutions to supervise a student view that observes only the qu…