Aligning to What? Limits to RLHF Based Alignment

Logan Barnhart, Reza Akbarian Bafghi, Stephen Becker +1

Reinforcement Learning from Human Feedback (RLHF) is increasingly used to align large language models (LLMs) with human preferences. However, the effectiveness of RLHF in addressin…