RISE: Recursive Improvement via Self-Extrapolating Policy Distillation

Yang Li, Semih Yavuz, Shafiq Joty

On-policy distillation (OPD) provides dense, per-token supervision for language model post-training, but its effectiveness is bottlenecked by teacher quality: external teachers suf…