Understanding the Role of Prompt Template in Knowledge Distillation for Safety Alignment

Anjila Budathoki, Manish Dhakal, Benjamin M. Ampel +1

Prior research has demonstrated that the choice of prompt template during Supervised Fine-Tuning (SFT) significantly impacts the robustness of safety alignment afterwards. However,…