Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Influence-Directed Distillation: Solving the Diversity Bottleneck in Sampled-Token On-Policy Distillation
Run Yang, Runpeng Dai, Jie Sun +5
Sampled-token on-policy distillation (OPD) efficiently transfers capabilities from teacher to student using student-generated tokens, requiring teacher probabilities only for sampl…
cs.CL2026
Large Language Model as Token Compressor and Decompressor
Wenbing Li, Yiran Wang, Zikai Song +4
In this paper, we study whether an off-the-shelf LLM can be adapted into a discrete, variable-length token compressor and decompressor for long-context processing. To this end, we…