Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
When Correct Demonstrations Hurt: Rethinking the Role of Exemplars in In-Context Learning
Chenghao Qiu, Chunli Peng, Yufeng Yang +2
In-context learning (ICL) is often motivated by the intuition that demonstrations help because they provide correct input-output examples. However, we reveal a counterintuitive phe…
cs.LG2026
Entropy-Aware On-Policy Distillation of Language Models
Woogyeol Jin, Taywon Min, Yongjin Yang +5
On-policy distillation is a promising approach for transferring knowledge between language models, where a student learns from dense token-level signals along its own trajectories.…