Margin-aware Preference Optimization for Aligning Diffusion Models without Reference

Jiwoo Hong, Sayak Paul, Noah Lee +3

Modern preference alignment methods, such as DPO, rely on divergence regularization to a reference model for training stability-but this creates a fundamental problem we call "refe…