Arena as Offline Reward: Efficient Fine-Grained Preference Optimization for Diffusion Models

Zhikai Li, Yue Zhao, Edward Zhongwei Zhang +4

Reinforcement learning from human feedback (RLHF) effectively promotes preference alignment of text-to-image (T2I) diffusion models. To improve computational efficiency, direct pre…