LLM Misalignment via Adversarial RLHF Platforms

Erfan Entezami, Ali Naseh

Reinforcement learning has shown remarkable performance in aligning language models with human preferences, leading to the rise of attention towards developing RLHF platforms. Thes…