Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF

Eric Zhu, Abhinav Shrivastava, Soumik Mukhopadhyay

Reinforcement learning from human feedback (RLHF) has emerged as a powerful paradigm for aligning generative models with human preferences. However, applying RLHF to diffusion mode…