2 papers
cs.LG2026
Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
Mohammadreza Armandpour, Fatih Ilhan, David Harrison +6
On-policy distillation offers dense, per-token supervision for training reasoning models; however, it remains unclear under which conditions this signal is beneficial and under whi…
cs.CL2025
Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential
Mohammad Samragh, Arnav Kundu, David Harrison +4
Autoregressive language models are constrained by their inherently sequential nature, generating one token at a time. This paradigm limits inference speed and parallelism, especial…