When Data is the Algorithm: A Systematic Study and Curation of Preference Optimization Datasets

Aladin Djuhera, Farhan Ahmed, Swanand Ravindra Kadhe +3

Aligning large language models (LLMs) is a central objective of post-training, often achieved through reward modeling and reinforcement learning methods. Among these, direct prefer…