Democratic Preference Alignment via Sortition-Weighted RLHF

Suvadip Sana, Jinzhou Wu, Martin T. Wells

Whose values should AI systems learn? Preference based alignment methods like RLHF derive their training signal from human raters, yet these rater pools are typically convenience s…