Addressing Over-Refusal in LLMs with Competing Rewards

Taeyoun Kim, Aviral Kumar

Safety training on language models often induces over-refusal: improved safety on harmful prompts at the cost of increased refusal on harmless ones. Though this trade-off can be mi…