Provable Zeroth-Order Safe Exploration for Reinforcement Learning from Human Feedback

Weiqin Chen, Xinmeng Huang, Dongsheng Ding +1

Safe reinforcement learning from human feedback (Safe-RLHF) seeks to improve helpfulness while satisfying harmlessness constraints. Existing primal-dual approaches can converge to…