Which Pairs to Compare for LLM Post-Training?

Jiangze Han, Vineet Goyal, Will Ma

Preference-based post-training has become a central paradigm for aligning language models. A common data-collection strategy is to generate a small set of completions for each prom…