Policy Gradient Primal-Dual Method for Safe Reinforcement Learning from Human Feedback

Qiang Liu, Adrienne Kline, Ermin Wei

Safe Reinforcement Learning from Human Feedback (Safe RLHF) has recently achieved empirical success in developing helpful and harmless large language models by decoupling human pre…