works on
large language models 1on-policy distillation 1rollout optimization 1structured pruning 1text generation 1

From the 1 of 1 linked paper with an AI index.

collaborators
Showing cs.LGShow all

1 paper · 1 filter

cs.LG2026

ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation

Qingyu Zhang, Qianhao Yuan, Hongyu Lin +7

The paper proposes ShortOPD, a short-to-long on-policy distillation method that recovers the generation quality of structured-pruned large language models by focusing training on e…