Showing cs.CLShow all
2 papers · 1 filter
cs.CL2026
Understanding the Role of Prompt Template in Knowledge Distillation for Safety Alignment
Anjila Budathoki, Manish Dhakal, Benjamin M. Ampel +1
Prior research has demonstrated that the choice of prompt template during Supervised Fine-Tuning (SFT) significantly impacts the robustness of safety alignment afterwards. However,…
cs.CL2026
DistillLens: Symmetric Knowledge Distillation Through Logit Lens
Manish Dhakal, Uthman Jinadu, Anjila Budathoki +2
Standard Knowledge Distillation (KD) compresses Large Language Models (LLMs) by optimizing final outputs, yet it typically treats the teacher's intermediate layer's thought process…