Rethinking harmless refusals when fine-tuning foundation models

Florin Pop, Judd Rosenblatt, Diogo Schwerz de Lucena +1

In this paper, we investigate the degree to which fine-tuning in Large Language Models (LLMs) effectively mitigates versus merely conceals undesirable behavior. Through the lens of…