SAFER: Probing Safety in Reward Models with Sparse Autoencoder

Wei Shi, Ziyuan Xie, Sihang Li +1

Reinforcement learning from human feedback (RLHF) is a key paradigm for aligning large language models (LLMs) with human values, yet the reward models at its core remain largely op…