SteerRM: Debiasing Reward Models via Sparse Autoencoders

Mengyuan Sun, Zhuohao Yu, Weizheng Gu +2

Reward models (RMs) are critical components of alignment pipelines, yet they exhibit biases toward superficial stylistic cues, preferring better-presented responses over semantical…