Picky LLMs and Unreliable RMs: An Empirical Study on Safety Alignment after Instruction Tuning

Guanlin Li, Kangjie Chen, Shangwei Guo +6

Large language models (LLMs) have emerged as powerful tools for addressing a wide range of general inquiries and tasks. Despite this, fine-tuning aligned LLMs on smaller, domain-sp…