Cat-DPO: Category-Adaptive Safety Alignment

Tiankai Yang, Yi Nian, Xinyuan Li +6

Aligning large language models with human preferences must balance two competing goals: responding helpfully to legitimate requests and reliably refusing harmful ones. Most prefere…