Users as Annotators: LLM Preference Learning from Comparison Mode

Zhongze Cai, Xiaocheng Li

Pairwise preference data have played an important role in the alignment of large language models (LLMs). Each sample of such data consists of a prompt, two different responses to t…