Evaluating Defences against Unsafe Feedback in RLHF

Domenic Rosati, Giles Edkins, Harsh Raj +5

While there has been progress towards aligning Large Language Models (LLMs) with human values and ensuring safe behaviour at inference time, safety guards can easily be removed whe…