BA-DPO: Bias-Adjusted Direct Preference Optimization for Language Model Alignment

Antonio Ferrara, Alberto Rumi, Francesco Bonchi

Preference-based alignment methods such as Direct Preference Optimization (DPO) use pairwise preferences labeled by human annotators to fine-tune language models. However, annotato…