Nearly Optimal Active Preference Learning and Its Application to LLM Alignment

Yao Zhao, Kwang-Sung Jun

Aligning large language models (LLMs) depends on high-quality datasets of human preference labels, which are costly to collect. Although active learning has been studied to improve…