Efficient RLHF: Reducing the Memory Usage of PPO

Michael Santacroce, Yadong Lu, Han Yu +2

Reinforcement Learning with Human Feedback (RLHF) has revolutionized language modeling by aligning models with human preferences. However, the RL stage, Proximal Policy Optimizatio…