A Geometric Perspective on Stabilizing Value Conflict Resolution

Saket Reddy, Andy Liu

Large Language Models (LLMs) often struggle to navigate value conflicts when trained with the compressed scalar rewards of Reinforcement Learning from Human Feedback (RLHF). To add…