Fairness Aware Reward Optimization

Ching Lam Choi, Vighnesh Subramaniam, Phillip Isola +2

Demographic skews in human preference data propagate systematic unfairness through reward models into aligned LLMs. We introduce Fairness Aware Reward Optimization (Faro), an in-pr…