2 papers
cs.LG2026
Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
Mohammadreza Armandpour, Fatih Ilhan, David Harrison +6
On-policy distillation offers dense, per-token supervision for training reasoning models; however, it remains unclear under which conditions this signal is beneficial and under whi…
cs.LG2025
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining
Jeffrey Li, Mohammadreza Armandpour, Iman Mirzadeh +8
Large Language Models (LLMs) trained on historical web data inevitably become outdated. We investigate evaluation strategies and update methods for LLMs as new data becomes availab…