Targeting Misalignment: A Conflict-Aware Framework for Reward-Model-based LLM Alignment

Zixuan Liu, Siavash H. Khajavi, Guangkai Jiang +1

Reward-model-based fine-tuning is a central paradigm in aligning Large Language Models with human preferences. However, such approaches critically rely on the assumption that proxy…