Beyond Ordinal Preferences: Why Alignment Needs Cardinal Human Feedback

Parker Whitfill, Stewy Slocum

Alignment techniques for LLMs rely on optimizing preference-based objectives -- where these preferences are typically elicited as ordinal, binary choices between responses. Recent…