The Unequal Influence of Bad Advice: Using Training Data Attribution to Modulate Emergent Misalignment

Gonçalo Paulo, Louis Jaburi, Nora Belrose +2

Fine-tuning large language models on narrow, misaligned tasks can undo their post-training alignment and induce novel misaligned behaviors -- a phenomenon known as \emph{emergent m…