EvoOPD: Evolving Privileged Information Strategies for Black-Box On-Policy Distillation

Ximo Zhu, Ruiqi Liu, Yi Han +6

Proprietary language models exhibit strong reasoning and coding capabilities, but their output distributions are inaccessible, making fine-grained distillation methods that rely on…