Rejected Dialects: Biases Against African American Language in Reward Models

Joel Mire, Zubin Trivadi Aysola, Daniel Chechelnitsky +3

Preference alignment via reward models helps build safe, helpful, and reliable large language models (LLMs). However, subjectivity in preference judgments and the lack of represent…