Efficient Preference Poisoning Attack on Offline RLHF

Chenye Yang, Weiyu Xu, Lifeng Lai

Offline Reinforcement Learning from Human Feedback (RLHF) pipelines such as Direct Preference Optimization (DPO) train on a pre-collected preference dataset, which makes them vulne…