SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks

Fenia Christopoulou, Ronald Cardenas, Gerasimos Lampouras +2

Direct alignment algorithms have proven an effective step for aligning language models to human-desired behaviors. Current variants of the Direct Preference Optimization objective…